Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.15000000000000002,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2100,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2100,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.654930
Epoch 1
Validation auc = 0.747418
Epoch 2
Validation auc = 0.767136
Epoch 3
Validation auc = 0.773709
Epoch 4
Loss = 1.4711e+00, PNorm = 82.4307, GNorm = 16.6891, lr_0 = 6.5000e-04
Validation auc = 0.749296
Epoch 5
Validation auc = 0.765258
Epoch 6
Validation auc = 0.790610
Epoch 7
Validation auc = 0.789202
Epoch 8
Validation auc = 0.784977
Epoch 9
Loss = 8.6202e-01, PNorm = 82.6871, GNorm = 29.2143, lr_0 = 9.9743e-04
Validation auc = 0.765728
Epoch 10
Validation auc = 0.755399
Epoch 11
Validation auc = 0.735211
Epoch 12
Validation auc = 0.744131
Epoch 13
Validation auc = 0.751643
Epoch 14
Loss = 7.4448e-01, PNorm = 82.9132, GNorm = 11.4339, lr_0 = 9.8890e-04
Validation auc = 0.769484
Epoch 15
Validation auc = 0.712207
Epoch 16
Validation auc = 0.728638
Epoch 17
Validation auc = 0.742254
Epoch 18
Validation auc = 0.702817
Epoch 19
Loss = 7.8800e-01, PNorm = 83.0566, GNorm = 14.6660, lr_0 = 9.8045e-04
Validation auc = 0.716901
Epoch 20
Validation auc = 0.762441
Epoch 21
Validation auc = 0.769484
Epoch 22
Validation auc = 0.772300
Epoch 23
Validation auc = 0.772300
Epoch 24
Loss = 7.6192e-01, PNorm = 83.1756, GNorm = 30.1798, lr_0 = 9.7207e-04
Validation auc = 0.770423
Epoch 25
Validation auc = 0.755399
Epoch 26
Validation auc = 0.752582
Epoch 27
Validation auc = 0.773239
Epoch 28
Validation auc = 0.761502
Epoch 29
Loss = 1.0140e+00, PNorm = 83.3220, GNorm = 28.9982, lr_0 = 9.6376e-04
Validation auc = 0.775587
Epoch 30
Validation auc = 0.768545
Epoch 31
Validation auc = 0.761972
Epoch 32
Validation auc = 0.751643
Epoch 33
Validation auc = 0.756338
Epoch 34
Loss = 5.6189e-01, PNorm = 83.4594, GNorm = 32.9679, lr_0 = 9.5552e-04
Validation auc = 0.747887
Epoch 35
Validation auc = 0.742488
Epoch 36
Validation auc = 0.722535
Epoch 37
Validation auc = 0.746009
Epoch 38
Validation auc = 0.741315
Epoch 39
Loss = 4.7680e-01, PNorm = 83.6398, GNorm = 18.0323, lr_0 = 9.4735e-04
Validation auc = 0.733803
Epoch 40
Validation auc = 0.737089
Epoch 41
Validation auc = 0.749296
Epoch 42
Validation auc = 0.747887
Epoch 43
Validation auc = 0.749296
Epoch 44
Loss = 3.5075e-01, PNorm = 83.7960, GNorm = 5.3719, lr_0 = 9.3925e-04
Validation auc = 0.748357
Epoch 45
Validation auc = 0.755399
Epoch 46
Validation auc = 0.775117
Epoch 47
Validation auc = 0.772770
Epoch 48
Validation auc = 0.769014
Epoch 49
Loss = 4.7472e-01, PNorm = 83.9785, GNorm = 11.1433, lr_0 = 9.3122e-04
Validation auc = 0.792488
Epoch 50
Validation auc = 0.793897
Epoch 51
Validation auc = 0.791549
Epoch 52
Validation auc = 0.759624
Epoch 53
Validation auc = 0.765258
Epoch 54
Loss = 6.5322e-01, PNorm = 84.1805, GNorm = 21.7712, lr_0 = 9.2326e-04
Validation auc = 0.766667
Epoch 55
Validation auc = 0.756808
Epoch 56
Validation auc = 0.771362
Epoch 57
Validation auc = 0.773239
Epoch 58
Validation auc = 0.775587
Epoch 59
Loss = 3.2341e-01, PNorm = 84.4564, GNorm = 5.7053, lr_0 = 9.1537e-04
Validation auc = 0.784977
Epoch 60
Validation auc = 0.783568
Epoch 61
Validation auc = 0.768545
Epoch 62
Validation auc = 0.768075
Epoch 63
Validation auc = 0.763850
Epoch 64
Loss = 3.3301e-01, PNorm = 84.7566, GNorm = 7.5642, lr_0 = 9.0754e-04
Validation auc = 0.757746
Epoch 65
Validation auc = 0.756338
Epoch 66
Validation auc = 0.745070
Epoch 67
Validation auc = 0.761972
Epoch 68
Validation auc = 0.775587
Epoch 69
Loss = 2.8203e-01, PNorm = 85.0370, GNorm = 6.8270, lr_0 = 8.9978e-04
Validation auc = 0.779343
Epoch 70
Validation auc = 0.798122
Epoch 71
Validation auc = 0.801408
Epoch 72
Validation auc = 0.801878
Epoch 73
Validation auc = 0.783568
Epoch 74
Loss = 1.6651e-01, PNorm = 85.2891, GNorm = 3.5549, lr_0 = 8.9209e-04
Validation auc = 0.747887
Epoch 75
Validation auc = 0.749296
Epoch 76
Validation auc = 0.746479
Epoch 77
Validation auc = 0.746009
Epoch 78
Validation auc = 0.776056
Epoch 79
Loss = 3.0929e-01, PNorm = 85.5517, GNorm = 7.2394, lr_0 = 8.8447e-04
Validation auc = 0.787324
Epoch 80
Validation auc = 0.778404
Epoch 81
Validation auc = 0.757277
Epoch 82
Validation auc = 0.756808
Epoch 83
Validation auc = 0.757746
Epoch 84
Loss = 3.6842e-01, PNorm = 85.8632, GNorm = 17.9401, lr_0 = 8.7691e-04
Validation auc = 0.769484
Epoch 85
Validation auc = 0.771362
Epoch 86
Validation auc = 0.788263
Epoch 87
Validation auc = 0.784507
Epoch 88
Validation auc = 0.788732
Epoch 89
Loss = 3.0535e-01, PNorm = 86.2085, GNorm = 12.5087, lr_0 = 8.6941e-04
Validation auc = 0.778404
Epoch 90
Validation auc = 0.782160
Epoch 91
Validation auc = 0.778873
Epoch 92
Validation auc = 0.790610
Epoch 93
Validation auc = 0.794366
Epoch 94
Loss = 2.3386e-01, PNorm = 86.5361, GNorm = 10.0391, lr_0 = 8.6198e-04
Validation auc = 0.794836
Epoch 95
Validation auc = 0.784977
Epoch 96
Validation auc = 0.778873
Epoch 97
Validation auc = 0.761502
Epoch 98
Validation auc = 0.746009
Epoch 99
Loss = 2.8820e-01, PNorm = 86.8168, GNorm = 8.9231, lr_0 = 8.5461e-04
Validation auc = 0.759155
Epoch 100
Validation auc = 0.763380
Epoch 101
Validation auc = 0.770892
Epoch 102
Validation auc = 0.772300
Epoch 103
Validation auc = 0.766667
Epoch 104
Loss = 2.6601e-01, PNorm = 87.1446, GNorm = 4.7378, lr_0 = 8.4730e-04
Validation auc = 0.760563
Epoch 105
Validation auc = 0.757277
Epoch 106
Validation auc = 0.758685
Epoch 107
Validation auc = 0.757746
Epoch 108
Validation auc = 0.763850
Epoch 109
Loss = 1.7656e-01, PNorm = 87.4878, GNorm = 3.6889, lr_0 = 8.4006e-04
Validation auc = 0.775117
Epoch 110
Validation auc = 0.779343
Epoch 111
Validation auc = 0.779812
Epoch 112
Validation auc = 0.777465
Epoch 113
Validation auc = 0.782629
Epoch 114
Loss = 2.5999e-01, PNorm = 87.8087, GNorm = 12.2839, lr_0 = 8.3288e-04
Validation auc = 0.788732
Epoch 115
Validation auc = 0.788263
Epoch 116
Validation auc = 0.800939
Epoch 117
Validation auc = 0.796714
Epoch 118
Validation auc = 0.793897
Epoch 119
Loss = 1.2763e-01, PNorm = 88.1346, GNorm = 2.6867, lr_0 = 8.2576e-04
Validation auc = 0.786385
Epoch 120
Validation auc = 0.783568
Epoch 121
Validation auc = 0.777465
Epoch 122
Validation auc = 0.774648
Epoch 123
Validation auc = 0.775117
Epoch 124
Loss = 1.6542e-01, PNorm = 88.4588, GNorm = 2.8912, lr_0 = 8.1870e-04
Validation auc = 0.768075
Epoch 125
Validation auc = 0.769014
Epoch 126
Validation auc = 0.762911
Epoch 127
Validation auc = 0.769014
Epoch 128
Validation auc = 0.785446
Epoch 129
Loss = 7.3849e-02, PNorm = 88.7646, GNorm = 3.6694, lr_0 = 8.1170e-04
Validation auc = 0.791080
Epoch 130
Validation auc = 0.795775
Epoch 131
Validation auc = 0.778404
Epoch 132
Validation auc = 0.764789
Epoch 133
Validation auc = 0.755399
Epoch 134
Loss = 9.6826e-02, PNorm = 89.0686, GNorm = 2.8896, lr_0 = 8.0476e-04
Validation auc = 0.761502
Epoch 135
Validation auc = 0.753052
Epoch 136
Validation auc = 0.756808
Epoch 137
Validation auc = 0.762441
Epoch 138
Validation auc = 0.767606
Epoch 139
Loss = 1.5486e-01, PNorm = 89.3797, GNorm = 2.0447, lr_0 = 7.9788e-04
Validation auc = 0.768075
Epoch 140
Validation auc = 0.773709
Epoch 141
Validation auc = 0.771362
Epoch 142
Validation auc = 0.750235
Epoch 143
Validation auc = 0.730986
Epoch 144
Loss = 1.1535e-01, PNorm = 89.7712, GNorm = 7.0019, lr_0 = 7.9106e-04
Validation auc = 0.729577
Epoch 145
Validation auc = 0.733803
Epoch 146
Validation auc = 0.739437
Epoch 147
Validation auc = 0.763850
Epoch 148
Validation auc = 0.781221
Epoch 149
Loss = 1.2098e-01, PNorm = 90.1046, GNorm = 3.5615, lr_0 = 7.8430e-04
Validation auc = 0.781690
Epoch 150
Validation auc = 0.786385
Epoch 151
Validation auc = 0.781690
Epoch 152
Validation auc = 0.776995
Epoch 153
Validation auc = 0.767606
Epoch 154
Loss = 1.2489e-01, PNorm = 90.3802, GNorm = 3.1060, lr_0 = 7.7759e-04
Validation auc = 0.767606
Epoch 155
Validation auc = 0.755869
Epoch 156
Validation auc = 0.743192
Epoch 157
Validation auc = 0.751174
Epoch 158
Validation auc = 0.756338
Epoch 159
Loss = 1.4870e-01, PNorm = 90.7080, GNorm = 8.1330, lr_0 = 7.7095e-04
Validation auc = 0.766667
Epoch 160
Validation auc = 0.769014
Epoch 161
Validation auc = 0.779343
Epoch 162
Validation auc = 0.787793
Epoch 163
Validation auc = 0.795775
Epoch 164
Loss = 1.3738e-01, PNorm = 91.0275, GNorm = 7.3108, lr_0 = 7.6436e-04
Validation auc = 0.794836
Epoch 165
Validation auc = 0.780282
Epoch 166
Validation auc = 0.769484
Epoch 167
Validation auc = 0.768545
Epoch 168
Validation auc = 0.775587
Epoch 169
Loss = 1.1154e-01, PNorm = 91.3017, GNorm = 6.5647, lr_0 = 7.5782e-04
Validation auc = 0.778873
Epoch 170
Validation auc = 0.776526
Epoch 171
Validation auc = 0.778404
Epoch 172
Validation auc = 0.781221
Epoch 173
Validation auc = 0.783099
Epoch 174
Loss = 1.6756e-01, PNorm = 91.5257, GNorm = 4.7258, lr_0 = 7.5134e-04
Validation auc = 0.783568
Epoch 175
Validation auc = 0.784038
Epoch 176
Validation auc = 0.784507
Epoch 177
Validation auc = 0.781690
Epoch 178
Validation auc = 0.780751
Epoch 179
Loss = 9.5334e-02, PNorm = 91.7234, GNorm = 4.5214, lr_0 = 7.4492e-04
Validation auc = 0.785446
Epoch 180
Validation auc = 0.792488
Epoch 181
Validation auc = 0.793897
Epoch 182
Validation auc = 0.793897
Epoch 183
Validation auc = 0.790141
Epoch 184
Loss = 4.3000e-02, PNorm = 91.9097, GNorm = 3.7683, lr_0 = 7.3855e-04
Validation auc = 0.792019
Epoch 185
Validation auc = 0.789671
Epoch 186
Validation auc = 0.790141
Epoch 187
Validation auc = 0.797183
Epoch 188
Validation auc = 0.801408
Epoch 189
Loss = 4.9031e-02, PNorm = 92.1445, GNorm = 1.7517, lr_0 = 7.3224e-04
Validation auc = 0.801878
Epoch 190
Validation auc = 0.805164
Epoch 191
Validation auc = 0.796244
Epoch 192
Validation auc = 0.798122
Epoch 193
Validation auc = 0.798592
Epoch 194
Loss = 1.2516e-01, PNorm = 92.3637, GNorm = 1.9216, lr_0 = 7.2598e-04
Validation auc = 0.786854
Epoch 195
Validation auc = 0.784977
Epoch 196
Validation auc = 0.781690
Epoch 197
Validation auc = 0.784038
Epoch 198
Validation auc = 0.776526
Epoch 199
Loss = 6.1112e-02, PNorm = 92.5810, GNorm = 2.3686, lr_0 = 7.1977e-04
Validation auc = 0.778404
Epoch 200
Validation auc = 0.777465
Epoch 201
Validation auc = 0.775117
Epoch 202
Validation auc = 0.770423
Epoch 203
Validation auc = 0.773709
Epoch 204
Loss = 4.7086e-02, PNorm = 92.8066, GNorm = 1.6529, lr_0 = 7.1362e-04
Validation auc = 0.772300
Epoch 205
Validation auc = 0.776056
Epoch 206
Validation auc = 0.778873
Epoch 207
Validation auc = 0.778404
Epoch 208
Validation auc = 0.771362
Epoch 209
Loss = 1.7267e-01, PNorm = 93.0529, GNorm = 1.5270, lr_0 = 7.0752e-04
Validation auc = 0.790141
Epoch 210
Validation auc = 0.790610
Epoch 211
Validation auc = 0.803756
Epoch 212
Validation auc = 0.807512
Epoch 213
Validation auc = 0.794366
Epoch 214
Loss = 6.9600e-02, PNorm = 93.3495, GNorm = 5.1301, lr_0 = 7.0147e-04
Validation auc = 0.778404
Epoch 215
Validation auc = 0.762441
Epoch 216
Validation auc = 0.764319
Epoch 217
Validation auc = 0.761502
Epoch 218
Validation auc = 0.758216
Epoch 219
Loss = 3.7390e-02, PNorm = 93.7157, GNorm = 1.8983, lr_0 = 6.9548e-04
Validation auc = 0.753991
Epoch 220
Validation auc = 0.747887
Epoch 221
Validation auc = 0.744601
Epoch 222
Validation auc = 0.747418
Epoch 223
Validation auc = 0.751174
Epoch 224
Loss = 8.5390e-02, PNorm = 94.0088, GNorm = 3.6451, lr_0 = 6.8953e-04
Validation auc = 0.748357
Epoch 225
Validation auc = 0.751643
Epoch 226
Validation auc = 0.750235
Epoch 227
Validation auc = 0.755869
Epoch 228
Validation auc = 0.763850
Epoch 229
Loss = 3.5317e-02, PNorm = 94.2610, GNorm = 1.5252, lr_0 = 6.8364e-04
Validation auc = 0.769014
Epoch 230
Validation auc = 0.770423
Epoch 231
Validation auc = 0.773239
Epoch 232
Validation auc = 0.776056
Epoch 233
Validation auc = 0.773709
Epoch 234
Loss = 6.1869e-03, PNorm = 94.4383, GNorm = 0.5689, lr_0 = 6.7779e-04
Validation auc = 0.767606
Epoch 235
Validation auc = 0.770423
Epoch 236
Validation auc = 0.773239
Epoch 237
Validation auc = 0.774178
Epoch 238
Validation auc = 0.774178
Epoch 239
Loss = 3.3141e-02, PNorm = 94.5659, GNorm = 2.2671, lr_0 = 6.7200e-04
Validation auc = 0.773709
Epoch 240
Validation auc = 0.774178
Epoch 241
Validation auc = 0.769014
Epoch 242
Validation auc = 0.768545
Epoch 243
Validation auc = 0.769484
Epoch 244
Loss = 1.3929e-02, PNorm = 94.6825, GNorm = 0.7266, lr_0 = 6.6625e-04
Validation auc = 0.767606
Epoch 245
Validation auc = 0.768075
Epoch 246
Validation auc = 0.767136
Epoch 247
Validation auc = 0.763380
Epoch 248
Validation auc = 0.762441
Epoch 249
Loss = 3.8787e-02, PNorm = 94.8210, GNorm = 2.4242, lr_0 = 6.6056e-04
Validation auc = 0.761033
Epoch 250
Validation auc = 0.754930
Epoch 251
Validation auc = 0.752113
Epoch 252
Validation auc = 0.750235
Epoch 253
Validation auc = 0.752113
Epoch 254
Loss = 1.0393e-02, PNorm = 95.0119, GNorm = 0.7496, lr_0 = 6.5491e-04
Validation auc = 0.756338
Epoch 255
Validation auc = 0.753991
Epoch 256
Validation auc = 0.752113
Epoch 257
Validation auc = 0.753991
Epoch 258
Validation auc = 0.756808
Epoch 259
Loss = 1.5025e-02, PNorm = 95.1670, GNorm = 0.7453, lr_0 = 6.4931e-04
Validation auc = 0.756808
Epoch 260
Validation auc = 0.761033
Epoch 261
Validation auc = 0.761972
Epoch 262
Validation auc = 0.763380
Epoch 263
Validation auc = 0.762441
Epoch 264
Loss = 5.1558e-02, PNorm = 95.2808, GNorm = 3.5668, lr_0 = 6.4376e-04
Validation auc = 0.767136
Epoch 265
Validation auc = 0.770423
Epoch 266
Validation auc = 0.766667
Epoch 267
Validation auc = 0.764319
Epoch 268
Validation auc = 0.761502
Epoch 269
Loss = 4.6751e-03, PNorm = 95.4049, GNorm = 0.1805, lr_0 = 6.3826e-04
Validation auc = 0.758685
Epoch 270
Validation auc = 0.750704
Epoch 271
Validation auc = 0.756338
Epoch 272
Validation auc = 0.757746
Epoch 273
Validation auc = 0.758216
Epoch 274
Loss = 7.7299e-02, PNorm = 95.5556, GNorm = 4.9264, lr_0 = 6.3280e-04
Validation auc = 0.752113
Epoch 275
Validation auc = 0.757277
Epoch 276
Validation auc = 0.760094
Epoch 277
Validation auc = 0.764319
Epoch 278
Validation auc = 0.764789
Epoch 279
Loss = 2.3389e-02, PNorm = 95.7744, GNorm = 1.9288, lr_0 = 6.2739e-04
Validation auc = 0.767136
Epoch 280
Validation auc = 0.776056
Epoch 281
Validation auc = 0.787793
Epoch 282
Validation auc = 0.799061
Epoch 283
Validation auc = 0.800469
Epoch 284
Loss = 3.6619e-02, PNorm = 95.9680, GNorm = 2.7757, lr_0 = 6.2203e-04
Validation auc = 0.802817
Epoch 285
Validation auc = 0.805164
Epoch 286
Validation auc = 0.810798
Epoch 287
Validation auc = 0.812207
Epoch 288
Validation auc = 0.811737
Epoch 289
Loss = 2.1258e-02, PNorm = 96.1012, GNorm = 1.7892, lr_0 = 6.1671e-04
Validation auc = 0.810329
Epoch 290
Validation auc = 0.803286
Epoch 291
Validation auc = 0.795305
Epoch 292
Validation auc = 0.791549
Epoch 293
Validation auc = 0.786385
Epoch 294
Loss = 2.2065e-02, PNorm = 96.2280, GNorm = 0.9821, lr_0 = 6.1144e-04
Validation auc = 0.784977
Epoch 295
Validation auc = 0.781221
Epoch 296
Validation auc = 0.781690
Epoch 297
Validation auc = 0.779343
Epoch 298
Validation auc = 0.778404
Epoch 299
Loss = 2.0924e-02, PNorm = 96.3631, GNorm = 3.2918, lr_0 = 6.0621e-04
Validation auc = 0.776056
Model 0 best validation auc = 0.812207 on epoch 287
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.804335
Ensemble test auc = 0.804335
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.737089
Epoch 1
Validation auc = 0.733333
Epoch 2
Validation auc = 0.771362
Epoch 3
Validation auc = 0.766667
Epoch 4
Loss = 1.6006e+00, PNorm = 82.4263, GNorm = 47.1407, lr_0 = 6.5000e-04
Validation auc = 0.780986
Epoch 5
Validation auc = 0.776056
Epoch 6
Validation auc = 0.765728
Epoch 7
Validation auc = 0.780751
Epoch 8
Validation auc = 0.758216
Epoch 9
Loss = 1.5067e+00, PNorm = 82.6648, GNorm = 27.9351, lr_0 = 9.9743e-04
Validation auc = 0.757042
Epoch 10
Validation auc = 0.763380
Epoch 11
Validation auc = 0.772300
Epoch 12
Validation auc = 0.751408
Epoch 13
Validation auc = 0.763380
Epoch 14
Loss = 9.4628e-01, PNorm = 82.8893, GNorm = 27.1736, lr_0 = 9.8890e-04
Validation auc = 0.760563
Epoch 15
Validation auc = 0.711972
Epoch 16
Validation auc = 0.764319
Epoch 17
Validation auc = 0.776995
Epoch 18
Validation auc = 0.769014
Epoch 19
Loss = 9.4127e-01, PNorm = 83.0298, GNorm = 10.5164, lr_0 = 9.8045e-04
Validation auc = 0.788263
Epoch 20
Validation auc = 0.778873
Epoch 21
Validation auc = 0.732864
Epoch 22
Validation auc = 0.753052
Epoch 23
Validation auc = 0.783099
Epoch 24
Loss = 8.6976e-01, PNorm = 83.1329, GNorm = 18.9071, lr_0 = 9.7207e-04
Validation auc = 0.769953
Epoch 25
Validation auc = 0.784507
Epoch 26
Validation auc = 0.806573
Epoch 27
Validation auc = 0.821596
Epoch 28
Validation auc = 0.831455
Epoch 29
Loss = 4.6416e-01, PNorm = 83.2410, GNorm = 6.6581, lr_0 = 9.6376e-04
Validation auc = 0.818310
Epoch 30
Validation auc = 0.811268
Epoch 31
Validation auc = 0.796714
Epoch 32
Validation auc = 0.792488
Epoch 33
Validation auc = 0.785446
Epoch 34
Loss = 4.7797e-01, PNorm = 83.3834, GNorm = 7.4096, lr_0 = 9.5552e-04
Validation auc = 0.798592
Epoch 35
Validation auc = 0.767606
Epoch 36
Validation auc = 0.750235
Epoch 37
Validation auc = 0.741315
Epoch 38
Validation auc = 0.764789
Epoch 39
Loss = 4.5300e-01, PNorm = 83.5551, GNorm = 6.8837, lr_0 = 9.4735e-04
Validation auc = 0.775117
Epoch 40
Validation auc = 0.780282
Epoch 41
Validation auc = 0.791080
Epoch 42
Validation auc = 0.770423
Epoch 43
Validation auc = 0.723005
Epoch 44
Loss = 6.0513e-01, PNorm = 83.7557, GNorm = 8.3518, lr_0 = 9.3925e-04
Validation auc = 0.733333
Epoch 45
Validation auc = 0.751174
Epoch 46
Validation auc = 0.770423
Epoch 47
Validation auc = 0.777934
Epoch 48
Validation auc = 0.773709
Epoch 49
Loss = 4.4298e-01, PNorm = 83.9905, GNorm = 4.9797, lr_0 = 9.3122e-04
Validation auc = 0.752582
Epoch 50
Validation auc = 0.748826
Epoch 51
Validation auc = 0.747887
Epoch 52
Validation auc = 0.747418
Epoch 53
Validation auc = 0.747418
Epoch 54
Loss = 5.2974e-01, PNorm = 84.2199, GNorm = 25.4869, lr_0 = 9.2326e-04
Validation auc = 0.741315
Epoch 55
Validation auc = 0.759624
Epoch 56
Validation auc = 0.744131
Epoch 57
Validation auc = 0.774178
Epoch 58
Validation auc = 0.785915
Epoch 59
Loss = 3.7700e-01, PNorm = 84.4546, GNorm = 7.7581, lr_0 = 9.1537e-04
Validation auc = 0.786385
Epoch 60
Validation auc = 0.792958
Epoch 61
Validation auc = 0.787793
Epoch 62
Validation auc = 0.776995
Epoch 63
Validation auc = 0.792019
Epoch 64
Loss = 3.1305e-01, PNorm = 84.7143, GNorm = 8.5998, lr_0 = 9.0754e-04
Validation auc = 0.787793
Epoch 65
Validation auc = 0.784977
Epoch 66
Validation auc = 0.788263
Epoch 67
Validation auc = 0.795775
Epoch 68
Validation auc = 0.792958
Epoch 69
Loss = 2.4688e-01, PNorm = 84.9567, GNorm = 4.9867, lr_0 = 8.9978e-04
Validation auc = 0.793427
Epoch 70
Validation auc = 0.782629
Epoch 71
Validation auc = 0.772770
Epoch 72
Validation auc = 0.764319
Epoch 73
Validation auc = 0.769953
Epoch 74
Loss = 2.8306e-01, PNorm = 85.2381, GNorm = 17.0954, lr_0 = 8.9209e-04
Validation auc = 0.754930
Epoch 75
Validation auc = 0.766197
Epoch 76
Validation auc = 0.788732
Epoch 77
Validation auc = 0.797653
Epoch 78
Validation auc = 0.797653
Epoch 79
Loss = 2.2984e-01, PNorm = 85.5975, GNorm = 12.3954, lr_0 = 8.8447e-04
Validation auc = 0.797653
Epoch 80
Validation auc = 0.796244
Epoch 81
Validation auc = 0.773709
Epoch 82
Validation auc = 0.747887
Epoch 83
Validation auc = 0.749765
Epoch 84
Loss = 3.4768e-01, PNorm = 85.9999, GNorm = 18.4787, lr_0 = 8.7691e-04
Validation auc = 0.766197
Epoch 85
Validation auc = 0.760563
Epoch 86
Validation auc = 0.751643
Epoch 87
Validation auc = 0.772770
Epoch 88
Validation auc = 0.778404
Epoch 89
Loss = 2.7978e-01, PNorm = 86.4047, GNorm = 6.8334, lr_0 = 8.6941e-04
Validation auc = 0.767606
Epoch 90
Validation auc = 0.752582
Epoch 91
Validation auc = 0.766667
Epoch 92
Validation auc = 0.761502
Epoch 93
Validation auc = 0.745070
Epoch 94
Loss = 2.0210e-01, PNorm = 86.8396, GNorm = 5.4209, lr_0 = 8.6198e-04
Validation auc = 0.741315
Epoch 95
Validation auc = 0.750704
Epoch 96
Validation auc = 0.753521
Epoch 97
Validation auc = 0.765728
Epoch 98
Validation auc = 0.790610
Epoch 99
Loss = 1.5071e-01, PNorm = 87.2915, GNorm = 6.3851, lr_0 = 8.5461e-04
Validation auc = 0.804695
Epoch 100
Validation auc = 0.802347
Epoch 101
Validation auc = 0.799531
Epoch 102
Validation auc = 0.799531
Epoch 103
Validation auc = 0.787324
Epoch 104
Loss = 1.4120e-01, PNorm = 87.6647, GNorm = 3.3999, lr_0 = 8.4730e-04
Validation auc = 0.775117
Epoch 105
Validation auc = 0.768075
Epoch 106
Validation auc = 0.769014
Epoch 107
Validation auc = 0.775587
Epoch 108
Validation auc = 0.784507
Epoch 109
Loss = 2.6901e-01, PNorm = 88.0460, GNorm = 6.8978, lr_0 = 8.4006e-04
Validation auc = 0.777934
Epoch 110
Validation auc = 0.782629
Epoch 111
Validation auc = 0.784977
Epoch 112
Validation auc = 0.764319
Epoch 113
Validation auc = 0.751643
Epoch 114
Loss = 1.1837e-01, PNorm = 88.4678, GNorm = 10.9846, lr_0 = 8.3288e-04
Validation auc = 0.775117
Epoch 115
Validation auc = 0.787324
Epoch 116
Validation auc = 0.789671
Epoch 117
Validation auc = 0.800469
Epoch 118
Validation auc = 0.802347
Epoch 119
Loss = 1.1817e-01, PNorm = 88.8817, GNorm = 3.2288, lr_0 = 8.2576e-04
Validation auc = 0.798592
Epoch 120
Validation auc = 0.792019
Epoch 121
Validation auc = 0.777934
Epoch 122
Validation auc = 0.786385
Epoch 123
Validation auc = 0.770892
Epoch 124
Loss = 1.9287e-01, PNorm = 89.3001, GNorm = 7.3113, lr_0 = 8.1870e-04
Validation auc = 0.768075
Epoch 125
Validation auc = 0.776526
Epoch 126
Validation auc = 0.776526
Epoch 127
Validation auc = 0.775117
Epoch 128
Validation auc = 0.783099
Epoch 129
Loss = 1.5308e-01, PNorm = 89.7381, GNorm = 6.4824, lr_0 = 8.1170e-04
Validation auc = 0.784038
Epoch 130
Validation auc = 0.771831
Epoch 131
Validation auc = 0.765728
Epoch 132
Validation auc = 0.774648
Epoch 133
Validation auc = 0.788732
Epoch 134
Loss = 1.7754e-01, PNorm = 90.1841, GNorm = 4.2700, lr_0 = 8.0476e-04
Validation auc = 0.791549
Epoch 135
Validation auc = 0.790610
Epoch 136
Validation auc = 0.799061
Epoch 137
Validation auc = 0.808451
Epoch 138
Validation auc = 0.805164
Epoch 139
Loss = 8.5320e-02, PNorm = 90.6205, GNorm = 3.9877, lr_0 = 7.9788e-04
Validation auc = 0.782160
Epoch 140
Validation auc = 0.776526
Epoch 141
Validation auc = 0.783568
Epoch 142
Validation auc = 0.789671
Epoch 143
Validation auc = 0.798122
Epoch 144
Loss = 1.0570e-01, PNorm = 91.0568, GNorm = 3.0081, lr_0 = 7.9106e-04
Validation auc = 0.797183
Epoch 145
Validation auc = 0.795305
Epoch 146
Validation auc = 0.794366
Epoch 147
Validation auc = 0.781221
Epoch 148
Validation auc = 0.771362
Epoch 149
Loss = 1.3443e-01, PNorm = 91.4829, GNorm = 4.7593, lr_0 = 7.8430e-04
Validation auc = 0.759155
Epoch 150
Validation auc = 0.777465
Epoch 151
Validation auc = 0.776056
Epoch 152
Validation auc = 0.776526
Epoch 153
Validation auc = 0.791080
Epoch 154
Loss = 1.0049e-01, PNorm = 91.8567, GNorm = 3.8369, lr_0 = 7.7759e-04
Validation auc = 0.783568
Epoch 155
Validation auc = 0.775587
Epoch 156
Validation auc = 0.777465
Epoch 157
Validation auc = 0.790141
Epoch 158
Validation auc = 0.792488
Epoch 159
Loss = 3.8644e-02, PNorm = 92.1875, GNorm = 1.1051, lr_0 = 7.7095e-04
Validation auc = 0.793897
Epoch 160
Validation auc = 0.790141
Epoch 161
Validation auc = 0.790141
Epoch 162
Validation auc = 0.801408
Epoch 163
Validation auc = 0.803756
Epoch 164
Loss = 8.7955e-02, PNorm = 92.5288, GNorm = 4.2588, lr_0 = 7.6436e-04
Validation auc = 0.802817
Epoch 165
Validation auc = 0.807512
Epoch 166
Validation auc = 0.812676
Epoch 167
Validation auc = 0.815023
Epoch 168
Validation auc = 0.815493
Epoch 169
Loss = 1.0729e-01, PNorm = 92.8392, GNorm = 1.1416, lr_0 = 7.5782e-04
Validation auc = 0.804225
Epoch 170
Validation auc = 0.819718
Epoch 171
Validation auc = 0.822066
Epoch 172
Validation auc = 0.819718
Epoch 173
Validation auc = 0.813146
Epoch 174
Loss = 6.4343e-02, PNorm = 93.2380, GNorm = 3.9935, lr_0 = 7.5134e-04
Validation auc = 0.813146
Epoch 175
Validation auc = 0.811737
Epoch 176
Validation auc = 0.814554
Epoch 177
Validation auc = 0.806573
Epoch 178
Validation auc = 0.806573
Epoch 179
Loss = 1.0946e-01, PNorm = 93.6410, GNorm = 2.3521, lr_0 = 7.4492e-04
Validation auc = 0.796714
Epoch 180
Validation auc = 0.788732
Epoch 181
Validation auc = 0.782629
Epoch 182
Validation auc = 0.775587
Epoch 183
Validation auc = 0.769953
Epoch 184
Loss = 5.0432e-02, PNorm = 94.0213, GNorm = 2.6507, lr_0 = 7.3855e-04
Validation auc = 0.766667
Epoch 185
Validation auc = 0.769484
Epoch 186
Validation auc = 0.770892
Epoch 187
Validation auc = 0.768075
Epoch 188
Validation auc = 0.769484
Epoch 189
Loss = 6.1451e-02, PNorm = 94.3421, GNorm = 4.2659, lr_0 = 7.3224e-04
Validation auc = 0.770423
Epoch 190
Validation auc = 0.774178
Epoch 191
Validation auc = 0.778404
Epoch 192
Validation auc = 0.772770
Epoch 193
Validation auc = 0.771362
Epoch 194
Loss = 1.3450e-01, PNorm = 94.6483, GNorm = 4.8582, lr_0 = 7.2598e-04
Validation auc = 0.769953
Epoch 195
Validation auc = 0.764789
Epoch 196
Validation auc = 0.761033
Epoch 197
Validation auc = 0.764789
Epoch 198
Validation auc = 0.765258
Epoch 199
Loss = 9.3847e-02, PNorm = 94.9633, GNorm = 4.0762, lr_0 = 7.1977e-04
Validation auc = 0.768075
Epoch 200
Validation auc = 0.769953
Epoch 201
Validation auc = 0.777465
Epoch 202
Validation auc = 0.781221
Epoch 203
Validation auc = 0.789202
Epoch 204
Loss = 7.5707e-02, PNorm = 95.2829, GNorm = 3.8427, lr_0 = 7.1362e-04
Validation auc = 0.796714
Epoch 205
Validation auc = 0.799061
Epoch 206
Validation auc = 0.810329
Epoch 207
Validation auc = 0.810329
Epoch 208
Validation auc = 0.795775
Epoch 209
Loss = 9.6701e-02, PNorm = 95.5762, GNorm = 2.1220, lr_0 = 7.0752e-04
Validation auc = 0.799531
Epoch 210
Validation auc = 0.797653
Epoch 211
Validation auc = 0.792488
Epoch 212
Validation auc = 0.784977
Epoch 213
Validation auc = 0.782629
Epoch 214
Loss = 9.6706e-03, PNorm = 95.8210, GNorm = 1.1199, lr_0 = 7.0147e-04
Validation auc = 0.788732
Epoch 215
Validation auc = 0.798122
Epoch 216
Validation auc = 0.800000
Epoch 217
Validation auc = 0.806103
Epoch 218
Validation auc = 0.809390
Epoch 219
Loss = 4.2573e-02, PNorm = 96.0310, GNorm = 0.4303, lr_0 = 6.9548e-04
Validation auc = 0.815962
Epoch 220
Validation auc = 0.805634
Epoch 221
Validation auc = 0.799061
Epoch 222
Validation auc = 0.795775
Epoch 223
Validation auc = 0.789202
Epoch 224
Loss = 7.7183e-02, PNorm = 96.2452, GNorm = 3.6283, lr_0 = 6.8953e-04
Validation auc = 0.791549
Epoch 225
Validation auc = 0.791080
Epoch 226
Validation auc = 0.791080
Epoch 227
Validation auc = 0.790141
Epoch 228
Validation auc = 0.790141
Epoch 229
Loss = 3.7552e-02, PNorm = 96.4472, GNorm = 1.0663, lr_0 = 6.8364e-04
Validation auc = 0.789202
Epoch 230
Validation auc = 0.788263
Epoch 231
Validation auc = 0.789671
Epoch 232
Validation auc = 0.790610
Epoch 233
Validation auc = 0.792958
Epoch 234
Loss = 9.0242e-03, PNorm = 96.6328, GNorm = 0.4835, lr_0 = 6.7779e-04
Validation auc = 0.794366
Epoch 235
Validation auc = 0.799531
Epoch 236
Validation auc = 0.800939
Epoch 237
Validation auc = 0.797653
Epoch 238
Validation auc = 0.795775
Epoch 239
Loss = 5.0174e-02, PNorm = 96.8734, GNorm = 5.0405, lr_0 = 6.7200e-04
Validation auc = 0.793427
Epoch 240
Validation auc = 0.789202
Epoch 241
Validation auc = 0.785915
Epoch 242
Validation auc = 0.786385
Epoch 243
Validation auc = 0.786385
Epoch 244
Loss = 1.3667e-02, PNorm = 97.1077, GNorm = 2.7021, lr_0 = 6.6625e-04
Validation auc = 0.790141
Epoch 245
Validation auc = 0.789671
Epoch 246
Validation auc = 0.786854
Epoch 247
Validation auc = 0.787324
Epoch 248
Validation auc = 0.787324
Epoch 249
Loss = 1.7045e-02, PNorm = 97.2871, GNorm = 1.0313, lr_0 = 6.6056e-04
Validation auc = 0.788732
Epoch 250
Validation auc = 0.787793
Epoch 251
Validation auc = 0.790141
Epoch 252
Validation auc = 0.791549
Epoch 253
Validation auc = 0.788263
Epoch 254
Loss = 1.5771e-02, PNorm = 97.4121, GNorm = 0.4597, lr_0 = 6.5491e-04
Validation auc = 0.784977
Epoch 255
Validation auc = 0.785915
Epoch 256
Validation auc = 0.782629
Epoch 257
Validation auc = 0.781690
Epoch 258
Validation auc = 0.779812
Epoch 259
Loss = 1.6752e-02, PNorm = 97.5170, GNorm = 3.1908, lr_0 = 6.4931e-04
Validation auc = 0.780751
Epoch 260
Validation auc = 0.781690
Epoch 261
Validation auc = 0.779812
Epoch 262
Validation auc = 0.774178
Epoch 263
Validation auc = 0.773709
Epoch 264
Loss = 1.0338e-02, PNorm = 97.6328, GNorm = 0.2207, lr_0 = 6.4376e-04
Validation auc = 0.775117
Epoch 265
Validation auc = 0.769014
Epoch 266
Validation auc = 0.763850
Epoch 267
Validation auc = 0.766667
Epoch 268
Validation auc = 0.764789
Epoch 269
Loss = 1.9183e-02, PNorm = 97.7395, GNorm = 0.4150, lr_0 = 6.3826e-04
Validation auc = 0.762441
Epoch 270
Validation auc = 0.758685
Epoch 271
Validation auc = 0.763380
Epoch 272
Validation auc = 0.769953
Epoch 273
Validation auc = 0.777465
Epoch 274
Loss = 7.6774e-03, PNorm = 97.8448, GNorm = 0.6463, lr_0 = 6.3280e-04
Validation auc = 0.779812
Epoch 275
Validation auc = 0.783568
Epoch 276
Validation auc = 0.785915
Epoch 277
Validation auc = 0.788263
Epoch 278
Validation auc = 0.787793
Epoch 279
Loss = 7.3992e-03, PNorm = 97.9541, GNorm = 0.3458, lr_0 = 6.2739e-04
Validation auc = 0.782629
Epoch 280
Validation auc = 0.787793
Epoch 281
Validation auc = 0.788732
Epoch 282
Validation auc = 0.786385
Epoch 283
Validation auc = 0.786854
Epoch 284
Loss = 7.3464e-03, PNorm = 98.0395, GNorm = 1.1837, lr_0 = 6.2203e-04
Validation auc = 0.787793
Epoch 285
Validation auc = 0.786854
Epoch 286
Validation auc = 0.786854
Epoch 287
Validation auc = 0.788263
Epoch 288
Validation auc = 0.790141
Epoch 289
Loss = 7.0070e-03, PNorm = 98.0963, GNorm = 0.3883, lr_0 = 6.1671e-04
Validation auc = 0.786385
Epoch 290
Validation auc = 0.789202
Epoch 291
Validation auc = 0.789671
Epoch 292
Validation auc = 0.787793
Epoch 293
Validation auc = 0.785446
Epoch 294
Loss = 4.3725e-03, PNorm = 98.1744, GNorm = 0.2005, lr_0 = 6.1144e-04
Validation auc = 0.785446
Epoch 295
Validation auc = 0.783568
Epoch 296
Validation auc = 0.783099
Epoch 297
Validation auc = 0.782160
Epoch 298
Validation auc = 0.784977
Epoch 299
Loss = 6.8310e-03, PNorm = 98.2397, GNorm = 0.2957, lr_0 = 6.0621e-04
Validation auc = 0.787793
Model 0 best validation auc = 0.831455 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.824804
Ensemble test auc = 0.824804
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.723005
Epoch 1
Validation auc = 0.742723
Epoch 2
Validation auc = 0.749765
Epoch 3
Validation auc = 0.747183
Epoch 4
Loss = 1.5657e+00, PNorm = 82.4274, GNorm = 13.3892, lr_0 = 6.5000e-04
Validation auc = 0.766197
Epoch 5
Validation auc = 0.797653
Epoch 6
Validation auc = 0.797183
Epoch 7
Validation auc = 0.794366
Epoch 8
Validation auc = 0.807042
Epoch 9
Loss = 1.0741e+00, PNorm = 82.6594, GNorm = 11.3394, lr_0 = 9.9743e-04
Validation auc = 0.778404
Epoch 10
Validation auc = 0.728169
Epoch 11
Validation auc = 0.758685
Epoch 12
Validation auc = 0.777934
Epoch 13
Validation auc = 0.783099
Epoch 14
Loss = 1.1027e+00, PNorm = 82.8858, GNorm = 41.4218, lr_0 = 9.8890e-04
Validation auc = 0.763380
Epoch 15
Validation auc = 0.664554
Epoch 16
Validation auc = 0.730047
Epoch 17
Validation auc = 0.748826
Epoch 18
Validation auc = 0.716432
Epoch 19
Loss = 1.4676e+00, PNorm = 83.0437, GNorm = 28.5304, lr_0 = 9.8045e-04
Validation auc = 0.756338
Epoch 20
Validation auc = 0.754460
Epoch 21
Validation auc = 0.721127
Epoch 22
Validation auc = 0.796714
Epoch 23
Validation auc = 0.786385
Epoch 24
Loss = 6.3855e-01, PNorm = 83.1663, GNorm = 9.0783, lr_0 = 9.7207e-04
Validation auc = 0.775587
Epoch 25
Validation auc = 0.763850
Epoch 26
Validation auc = 0.770892
Epoch 27
Validation auc = 0.733803
Epoch 28
Validation auc = 0.706573
Epoch 29
Loss = 1.1129e+00, PNorm = 83.2840, GNorm = 21.7904, lr_0 = 9.6376e-04
Validation auc = 0.716901
Epoch 30
Validation auc = 0.722066
Epoch 31
Validation auc = 0.743192
Epoch 32
Validation auc = 0.760563
Epoch 33
Validation auc = 0.778873
Epoch 34
Loss = 8.0176e-01, PNorm = 83.4361, GNorm = 27.6983, lr_0 = 9.5552e-04
Validation auc = 0.804225
Epoch 35
Validation auc = 0.814085
Epoch 36
Validation auc = 0.792958
Epoch 37
Validation auc = 0.784038
Epoch 38
Validation auc = 0.776995
Epoch 39
Loss = 7.3880e-01, PNorm = 83.6241, GNorm = 24.0143, lr_0 = 9.4735e-04
Validation auc = 0.726291
Epoch 40
Validation auc = 0.715023
Epoch 41
Validation auc = 0.720657
Epoch 42
Validation auc = 0.754460
Epoch 43
Validation auc = 0.717371
Epoch 44
Loss = 8.4927e-01, PNorm = 83.8649, GNorm = 12.1274, lr_0 = 9.3925e-04
Validation auc = 0.737559
Epoch 45
Validation auc = 0.721127
Epoch 46
Validation auc = 0.712676
Epoch 47
Validation auc = 0.682160
Epoch 48
Validation auc = 0.667136
Epoch 49
Loss = 4.3760e-01, PNorm = 84.1710, GNorm = 6.3293, lr_0 = 9.3122e-04
Validation auc = 0.737089
Epoch 50
Validation auc = 0.753521
Epoch 51
Validation auc = 0.734742
Epoch 52
Validation auc = 0.750235
Epoch 53
Validation auc = 0.731455
Epoch 54
Loss = 5.1688e-01, PNorm = 84.4731, GNorm = 16.2334, lr_0 = 9.2326e-04
Validation auc = 0.723005
Epoch 55
Validation auc = 0.762911
Epoch 56
Validation auc = 0.769014
Epoch 57
Validation auc = 0.761033
Epoch 58
Validation auc = 0.776056
Epoch 59
Loss = 5.1478e-01, PNorm = 84.7559, GNorm = 26.6968, lr_0 = 9.1537e-04
Validation auc = 0.777934
Epoch 60
Validation auc = 0.776526
Epoch 61
Validation auc = 0.784977
Epoch 62
Validation auc = 0.783568
Epoch 63
Validation auc = 0.772770
Epoch 64
Loss = 3.4062e-01, PNorm = 84.9896, GNorm = 10.2483, lr_0 = 9.0754e-04
Validation auc = 0.746948
Epoch 65
Validation auc = 0.775117
Epoch 66
Validation auc = 0.765728
Epoch 67
Validation auc = 0.775587
Epoch 68
Validation auc = 0.775587
Epoch 69
Loss = 2.2764e-01, PNorm = 85.2332, GNorm = 11.3489, lr_0 = 8.9978e-04
Validation auc = 0.794366
Epoch 70
Validation auc = 0.798122
Epoch 71
Validation auc = 0.786385
Epoch 72
Validation auc = 0.785915
Epoch 73
Validation auc = 0.766197
Epoch 74
Loss = 2.2912e-01, PNorm = 85.5021, GNorm = 5.7443, lr_0 = 8.9209e-04
Validation auc = 0.757746
Epoch 75
Validation auc = 0.741784
Epoch 76
Validation auc = 0.756808
Epoch 77
Validation auc = 0.771831
Epoch 78
Validation auc = 0.783568
Epoch 79
Loss = 2.9881e-01, PNorm = 85.8701, GNorm = 6.8535, lr_0 = 8.8447e-04
Validation auc = 0.790141
Epoch 80
Validation auc = 0.792019
Epoch 81
Validation auc = 0.798122
Epoch 82
Validation auc = 0.790141
Epoch 83
Validation auc = 0.771831
Epoch 84
Loss = 2.8740e-01, PNorm = 86.1844, GNorm = 10.3783, lr_0 = 8.7691e-04
Validation auc = 0.769953
Epoch 85
Validation auc = 0.761033
Epoch 86
Validation auc = 0.753991
Epoch 87
Validation auc = 0.751174
Epoch 88
Validation auc = 0.763380
Epoch 89
Loss = 2.1440e-01, PNorm = 86.5964, GNorm = 4.9979, lr_0 = 8.6941e-04
Validation auc = 0.779812
Epoch 90
Validation auc = 0.780282
Epoch 91
Validation auc = 0.772300
Epoch 92
Validation auc = 0.766667
Epoch 93
Validation auc = 0.770423
Epoch 94
Loss = 2.5298e-01, PNorm = 87.0015, GNorm = 15.1376, lr_0 = 8.6198e-04
Validation auc = 0.766197
Epoch 95
Validation auc = 0.755869
Epoch 96
Validation auc = 0.755869
Epoch 97
Validation auc = 0.756808
Epoch 98
Validation auc = 0.767606
Epoch 99
Loss = 3.0947e-01, PNorm = 87.3583, GNorm = 18.5411, lr_0 = 8.5461e-04
Validation auc = 0.775117
Epoch 100
Validation auc = 0.792488
Epoch 101
Validation auc = 0.786385
Epoch 102
Validation auc = 0.773709
Epoch 103
Validation auc = 0.770423
Epoch 104
Loss = 2.4749e-01, PNorm = 87.7702, GNorm = 10.9725, lr_0 = 8.4730e-04
Validation auc = 0.769484
Epoch 105
Validation auc = 0.774178
Epoch 106
Validation auc = 0.751643
Epoch 107
Validation auc = 0.770892
Epoch 108
Validation auc = 0.773709
Epoch 109
Loss = 2.7426e-01, PNorm = 88.2118, GNorm = 10.9682, lr_0 = 8.4006e-04
Validation auc = 0.779343
Epoch 110
Validation auc = 0.799061
Epoch 111
Validation auc = 0.798592
Epoch 112
Validation auc = 0.793427
Epoch 113
Validation auc = 0.792019
Epoch 114
Loss = 2.9240e-01, PNorm = 88.6643, GNorm = 8.3998, lr_0 = 8.3288e-04
Validation auc = 0.785446
Epoch 115
Validation auc = 0.774648
Epoch 116
Validation auc = 0.762911
Epoch 117
Validation auc = 0.770423
Epoch 118
Validation auc = 0.772300
Epoch 119
Loss = 1.3444e-01, PNorm = 89.0784, GNorm = 3.6353, lr_0 = 8.2576e-04
Validation auc = 0.769953
Epoch 120
Validation auc = 0.774648
Epoch 121
Validation auc = 0.772770
Epoch 122
Validation auc = 0.783568
Epoch 123
Validation auc = 0.778873
Epoch 124
Loss = 2.1936e-01, PNorm = 89.4835, GNorm = 4.9092, lr_0 = 8.1870e-04
Validation auc = 0.767606
Epoch 125
Validation auc = 0.770423
Epoch 126
Validation auc = 0.777934
Epoch 127
Validation auc = 0.785915
Epoch 128
Validation auc = 0.783099
Epoch 129
Loss = 1.9468e-01, PNorm = 89.9199, GNorm = 4.3711, lr_0 = 8.1170e-04
Validation auc = 0.789671
Epoch 130
Validation auc = 0.792958
Epoch 131
Validation auc = 0.776995
Epoch 132
Validation auc = 0.772300
Epoch 133
Validation auc = 0.763850
Epoch 134
Loss = 9.7905e-02, PNorm = 90.3967, GNorm = 3.9986, lr_0 = 8.0476e-04
Validation auc = 0.760563
Epoch 135
Validation auc = 0.754460
Epoch 136
Validation auc = 0.749296
Epoch 137
Validation auc = 0.768075
Epoch 138
Validation auc = 0.776056
Epoch 139
Loss = 2.1022e-01, PNorm = 90.8004, GNorm = 14.1982, lr_0 = 7.9788e-04
Validation auc = 0.788732
Epoch 140
Validation auc = 0.795775
Epoch 141
Validation auc = 0.782160
Epoch 142
Validation auc = 0.768545
Epoch 143
Validation auc = 0.761972
Epoch 144
Loss = 1.2277e-01, PNorm = 91.1844, GNorm = 2.8938, lr_0 = 7.9106e-04
Validation auc = 0.768075
Epoch 145
Validation auc = 0.766667
Epoch 146
Validation auc = 0.773239
Epoch 147
Validation auc = 0.763850
Epoch 148
Validation auc = 0.757746
Epoch 149
Loss = 7.7201e-02, PNorm = 91.5873, GNorm = 1.9147, lr_0 = 7.8430e-04
Validation auc = 0.757746
Epoch 150
Validation auc = 0.757746
Epoch 151
Validation auc = 0.756808
Epoch 152
Validation auc = 0.755869
Epoch 153
Validation auc = 0.772770
Epoch 154
Loss = 1.6903e-01, PNorm = 91.9834, GNorm = 1.8446, lr_0 = 7.7759e-04
Validation auc = 0.791549
Epoch 155
Validation auc = 0.785446
Epoch 156
Validation auc = 0.780282
Epoch 157
Validation auc = 0.774178
Epoch 158
Validation auc = 0.776526
Epoch 159
Loss = 1.1296e-01, PNorm = 92.3716, GNorm = 2.6738, lr_0 = 7.7095e-04
Validation auc = 0.781221
Epoch 160
Validation auc = 0.783099
Epoch 161
Validation auc = 0.783099
Epoch 162
Validation auc = 0.780282
Epoch 163
Validation auc = 0.774648
Epoch 164
Loss = 9.0410e-02, PNorm = 92.7689, GNorm = 5.8685, lr_0 = 7.6436e-04
Validation auc = 0.776995
Epoch 165
Validation auc = 0.787793
Epoch 166
Validation auc = 0.793897
Epoch 167
Validation auc = 0.793427
Epoch 168
Validation auc = 0.809859
Epoch 169
Loss = 6.7384e-02, PNorm = 93.1293, GNorm = 3.1781, lr_0 = 7.5782e-04
Validation auc = 0.803756
Epoch 170
Validation auc = 0.802817
Epoch 171
Validation auc = 0.816432
Epoch 172
Validation auc = 0.825352
Epoch 173
Validation auc = 0.822066
Epoch 174
Loss = 1.8262e-01, PNorm = 93.5285, GNorm = 2.0693, lr_0 = 7.5134e-04
Validation auc = 0.814085
Epoch 175
Validation auc = 0.804695
Epoch 176
Validation auc = 0.804695
Epoch 177
Validation auc = 0.794366
Epoch 178
Validation auc = 0.780751
Epoch 179
Loss = 1.0819e-01, PNorm = 93.9302, GNorm = 5.4536, lr_0 = 7.4492e-04
Validation auc = 0.778873
Epoch 180
Validation auc = 0.791549
Epoch 181
Validation auc = 0.791549
Epoch 182
Validation auc = 0.795775
Epoch 183
Validation auc = 0.799061
Epoch 184
Loss = 1.1425e-01, PNorm = 94.3139, GNorm = 7.1518, lr_0 = 7.3855e-04
Validation auc = 0.791080
Epoch 185
Validation auc = 0.787793
Epoch 186
Validation auc = 0.768075
Epoch 187
Validation auc = 0.764789
Epoch 188
Validation auc = 0.773239
Epoch 189
Loss = 7.0708e-02, PNorm = 94.7286, GNorm = 4.4586, lr_0 = 7.3224e-04
Validation auc = 0.773239
Epoch 190
Validation auc = 0.773239
Epoch 191
Validation auc = 0.779343
Epoch 192
Validation auc = 0.784038
Epoch 193
Validation auc = 0.790610
Epoch 194
Loss = 5.0140e-02, PNorm = 95.0789, GNorm = 5.0890, lr_0 = 7.2598e-04
Validation auc = 0.795305
Epoch 195
Validation auc = 0.790141
Epoch 196
Validation auc = 0.783568
Epoch 197
Validation auc = 0.781221
Epoch 198
Validation auc = 0.777465
Epoch 199
Loss = 4.6540e-02, PNorm = 95.3922, GNorm = 1.8816, lr_0 = 7.1977e-04
Validation auc = 0.775117
Epoch 200
Validation auc = 0.770892
Epoch 201
Validation auc = 0.761502
Epoch 202
Validation auc = 0.758216
Epoch 203
Validation auc = 0.761268
Epoch 204
Loss = 9.4500e-02, PNorm = 95.6852, GNorm = 7.1905, lr_0 = 7.1362e-04
Validation auc = 0.768545
Epoch 205
Validation auc = 0.782629
Epoch 206
Validation auc = 0.795775
Epoch 207
Validation auc = 0.794836
Epoch 208
Validation auc = 0.806573
Epoch 209
Loss = 2.2768e-02, PNorm = 96.0478, GNorm = 1.8271, lr_0 = 7.0752e-04
Validation auc = 0.815962
Epoch 210
Validation auc = 0.816432
Epoch 211
Validation auc = 0.815962
Epoch 212
Validation auc = 0.820188
Epoch 213
Validation auc = 0.818310
Epoch 214
Loss = 2.2575e-02, PNorm = 96.3011, GNorm = 1.4829, lr_0 = 7.0147e-04
Validation auc = 0.810798
Epoch 215
Validation auc = 0.811268
Epoch 216
Validation auc = 0.809390
Epoch 217
Validation auc = 0.809390
Epoch 218
Validation auc = 0.808451
Epoch 219
Loss = 3.0133e-02, PNorm = 96.5330, GNorm = 1.5725, lr_0 = 6.9548e-04
Validation auc = 0.799061
Epoch 220
Validation auc = 0.791080
Epoch 221
Validation auc = 0.788263
Epoch 222
Validation auc = 0.780282
Epoch 223
Validation auc = 0.764789
Epoch 224
Loss = 3.5241e-02, PNorm = 96.7869, GNorm = 2.3289, lr_0 = 6.8953e-04
Validation auc = 0.758685
Epoch 225
Validation auc = 0.767136
Epoch 226
Validation auc = 0.767606
Epoch 227
Validation auc = 0.777934
Epoch 228
Validation auc = 0.779343
Epoch 229
Loss = 5.9982e-02, PNorm = 97.0118, GNorm = 0.7395, lr_0 = 6.8364e-04
Validation auc = 0.778404
Epoch 230
Validation auc = 0.782160
Epoch 231
Validation auc = 0.782160
Epoch 232
Validation auc = 0.779343
Epoch 233
Validation auc = 0.777934
Epoch 234
Loss = 2.9408e-02, PNorm = 97.2177, GNorm = 0.2530, lr_0 = 6.7779e-04
Validation auc = 0.769014
Epoch 235
Validation auc = 0.765728
Epoch 236
Validation auc = 0.770423
Epoch 237
Validation auc = 0.771362
Epoch 238
Validation auc = 0.779812
Epoch 239
Loss = 8.7965e-02, PNorm = 97.4264, GNorm = 8.5604, lr_0 = 6.7200e-04
Validation auc = 0.777465
Epoch 240
Validation auc = 0.784038
Epoch 241
Validation auc = 0.784038
Epoch 242
Validation auc = 0.777934
Epoch 243
Validation auc = 0.778404
Epoch 244
Loss = 8.6295e-02, PNorm = 97.6956, GNorm = 3.0064, lr_0 = 6.6625e-04
Validation auc = 0.776995
Epoch 245
Validation auc = 0.773239
Epoch 246
Validation auc = 0.771831
Epoch 247
Validation auc = 0.767136
Epoch 248
Validation auc = 0.766197
Epoch 249
Loss = 8.1789e-03, PNorm = 97.9831, GNorm = 0.7912, lr_0 = 6.6056e-04
Validation auc = 0.776526
Epoch 250
Validation auc = 0.783099
Epoch 251
Validation auc = 0.781690
Epoch 252
Validation auc = 0.781221
Epoch 253
Validation auc = 0.781690
Epoch 254
Loss = 2.7622e-02, PNorm = 98.3019, GNorm = 0.7755, lr_0 = 6.5491e-04
Validation auc = 0.780282
Epoch 255
Validation auc = 0.781690
Epoch 256
Validation auc = 0.768545
Epoch 257
Validation auc = 0.767136
Epoch 258
Validation auc = 0.771831
Epoch 259
Loss = 1.4246e-02, PNorm = 98.5649, GNorm = 1.5572, lr_0 = 6.4931e-04
Validation auc = 0.771831
Epoch 260
Validation auc = 0.769484
Epoch 261
Validation auc = 0.772770
Epoch 262
Validation auc = 0.773709
Epoch 263
Validation auc = 0.769484
Epoch 264
Loss = 5.8522e-03, PNorm = 98.7922, GNorm = 0.3002, lr_0 = 6.4376e-04
Validation auc = 0.766667
Epoch 265
Validation auc = 0.765728
Epoch 266
Validation auc = 0.774178
Epoch 267
Validation auc = 0.777465
Epoch 268
Validation auc = 0.780282
Epoch 269
Loss = 1.2591e-02, PNorm = 99.0067, GNorm = 0.8039, lr_0 = 6.3826e-04
Validation auc = 0.780282
Epoch 270
Validation auc = 0.781221
Epoch 271
Validation auc = 0.784507
Epoch 272
Validation auc = 0.784507
Epoch 273
Validation auc = 0.782629
Epoch 274
Loss = 1.7265e-02, PNorm = 99.1551, GNorm = 3.3075, lr_0 = 6.3280e-04
Validation auc = 0.777934
Epoch 275
Validation auc = 0.772300
Epoch 276
Validation auc = 0.765258
Epoch 277
Validation auc = 0.762911
Epoch 278
Validation auc = 0.761033
Epoch 279
Loss = 2.3658e-02, PNorm = 99.2698, GNorm = 3.6202, lr_0 = 6.2739e-04
Validation auc = 0.761972
Epoch 280
Validation auc = 0.757277
Epoch 281
Validation auc = 0.760094
Epoch 282
Validation auc = 0.760563
Epoch 283
Validation auc = 0.757277
Epoch 284
Loss = 4.1305e-02, PNorm = 99.3700, GNorm = 2.3544, lr_0 = 6.2203e-04
Validation auc = 0.755869
Epoch 285
Validation auc = 0.758216
Epoch 286
Validation auc = 0.757746
Epoch 287
Validation auc = 0.754930
Epoch 288
Validation auc = 0.753052
Epoch 289
Loss = 2.3654e-02, PNorm = 99.4751, GNorm = 4.4030, lr_0 = 6.1671e-04
Validation auc = 0.752582
Epoch 290
Validation auc = 0.752582
Epoch 291
Validation auc = 0.747887
Epoch 292
Validation auc = 0.747887
Epoch 293
Validation auc = 0.752582
Epoch 294
Loss = 7.2625e-03, PNorm = 99.5981, GNorm = 0.6315, lr_0 = 6.1144e-04
Validation auc = 0.757746
Epoch 295
Validation auc = 0.752582
Epoch 296
Validation auc = 0.753052
Epoch 297
Validation auc = 0.756338
Epoch 298
Validation auc = 0.758685
Epoch 299
Loss = 8.3516e-03, PNorm = 99.7169, GNorm = 1.5042, lr_0 = 6.0621e-04
Validation auc = 0.763850
Model 0 best validation auc = 0.825352 on epoch 172
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.778447
Ensemble test auc = 0.778447
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.736620
Epoch 1
Validation auc = 0.757277
Epoch 2
Validation auc = 0.761972
Epoch 3
Validation auc = 0.780282
Epoch 4
Loss = 1.4162e+00, PNorm = 82.4255, GNorm = 33.7511, lr_0 = 6.5000e-04
Validation auc = 0.807042
Epoch 5
Validation auc = 0.815023
Epoch 6
Validation auc = 0.802817
Epoch 7
Validation auc = 0.778404
Epoch 8
Validation auc = 0.780751
Epoch 9
Loss = 1.2395e+00, PNorm = 82.6640, GNorm = 38.7876, lr_0 = 9.9743e-04
Validation auc = 0.781690
Epoch 10
Validation auc = 0.766667
Epoch 11
Validation auc = 0.735681
Epoch 12
Validation auc = 0.667136
Epoch 13
Validation auc = 0.738967
Epoch 14
Loss = 1.0031e+00, PNorm = 82.8987, GNorm = 14.2957, lr_0 = 9.8890e-04
Validation auc = 0.748826
Epoch 15
Validation auc = 0.746948
Epoch 16
Validation auc = 0.752113
Epoch 17
Validation auc = 0.755399
Epoch 18
Validation auc = 0.766667
Epoch 19
Loss = 4.7113e-01, PNorm = 83.0716, GNorm = 8.2559, lr_0 = 9.8045e-04
Validation auc = 0.798592
Epoch 20
Validation auc = 0.785446
Epoch 21
Validation auc = 0.761972
Epoch 22
Validation auc = 0.755399
Epoch 23
Validation auc = 0.757277
Epoch 24
Loss = 8.5698e-01, PNorm = 83.2303, GNorm = 21.7321, lr_0 = 9.7207e-04
Validation auc = 0.744601
Epoch 25
Validation auc = 0.716432
Epoch 26
Validation auc = 0.730047
Epoch 27
Validation auc = 0.752582
Epoch 28
Validation auc = 0.730986
Epoch 29
Loss = 1.2314e+00, PNorm = 83.3889, GNorm = 12.1085, lr_0 = 9.6376e-04
Validation auc = 0.760563
Epoch 30
Validation auc = 0.755399
Epoch 31
Validation auc = 0.777465
Epoch 32
Validation auc = 0.752582
Epoch 33
Validation auc = 0.784977
Epoch 34
Loss = 5.2981e-01, PNorm = 83.6012, GNorm = 13.6564, lr_0 = 9.5552e-04
Validation auc = 0.773709
Epoch 35
Validation auc = 0.769484
Epoch 36
Validation auc = 0.769953
Epoch 37
Validation auc = 0.751878
Epoch 38
Validation auc = 0.750704
Epoch 39
Loss = 7.5589e-01, PNorm = 83.7591, GNorm = 27.9919, lr_0 = 9.4735e-04
Validation auc = 0.748357
Epoch 40
Validation auc = 0.752113
Epoch 41
Validation auc = 0.781690
Epoch 42
Validation auc = 0.712676
Epoch 43
Validation auc = 0.774178
Epoch 44
Loss = 1.0374e+00, PNorm = 83.9669, GNorm = 47.8157, lr_0 = 9.3925e-04
Validation auc = 0.786385
Epoch 45
Validation auc = 0.779343
Epoch 46
Validation auc = 0.777465
Epoch 47
Validation auc = 0.778404
Epoch 48
Validation auc = 0.784038
Epoch 49
Loss = 5.9016e-01, PNorm = 84.2026, GNorm = 8.9966, lr_0 = 9.3122e-04
Validation auc = 0.798122
Epoch 50
Validation auc = 0.783568
Epoch 51
Validation auc = 0.789202
Epoch 52
Validation auc = 0.789671
Epoch 53
Validation auc = 0.771831
Epoch 54
Loss = 5.5905e-01, PNorm = 84.4330, GNorm = 16.7309, lr_0 = 9.2326e-04
Validation auc = 0.757746
Epoch 55
Validation auc = 0.753521
Epoch 56
Validation auc = 0.766667
Epoch 57
Validation auc = 0.746009
Epoch 58
Validation auc = 0.728169
Epoch 59
Loss = 4.3892e-01, PNorm = 84.6921, GNorm = 13.3911, lr_0 = 9.1537e-04
Validation auc = 0.734742
Epoch 60
Validation auc = 0.746009
Epoch 61
Validation auc = 0.738028
Epoch 62
Validation auc = 0.746009
Epoch 63
Validation auc = 0.778404
Epoch 64
Loss = 4.4056e-01, PNorm = 84.9624, GNorm = 9.3188, lr_0 = 9.0754e-04
Validation auc = 0.781221
Epoch 65
Validation auc = 0.782629
Epoch 66
Validation auc = 0.785446
Epoch 67
Validation auc = 0.783568
Epoch 68
Validation auc = 0.754460
Epoch 69
Loss = 2.6924e-01, PNorm = 85.2840, GNorm = 11.6236, lr_0 = 8.9978e-04
Validation auc = 0.761972
Epoch 70
Validation auc = 0.752582
Epoch 71
Validation auc = 0.742254
Epoch 72
Validation auc = 0.754460
Epoch 73
Validation auc = 0.762441
Epoch 74
Loss = 2.2923e-01, PNorm = 85.5533, GNorm = 14.3520, lr_0 = 8.9209e-04
Validation auc = 0.764789
Epoch 75
Validation auc = 0.784038
Epoch 76
Validation auc = 0.784507
Epoch 77
Validation auc = 0.784038
Epoch 78
Validation auc = 0.790610
Epoch 79
Loss = 2.9131e-01, PNorm = 85.9099, GNorm = 6.2384, lr_0 = 8.8447e-04
Validation auc = 0.790610
Epoch 80
Validation auc = 0.784507
Epoch 81
Validation auc = 0.777465
Epoch 82
Validation auc = 0.785446
Epoch 83
Validation auc = 0.775117
Epoch 84
Loss = 1.9027e-01, PNorm = 86.2886, GNorm = 6.2626, lr_0 = 8.7691e-04
Validation auc = 0.770892
Epoch 85
Validation auc = 0.769953
Epoch 86
Validation auc = 0.777934
Epoch 87
Validation auc = 0.793897
Epoch 88
Validation auc = 0.802347
Epoch 89
Loss = 3.0272e-01, PNorm = 86.6445, GNorm = 13.6667, lr_0 = 8.6941e-04
Validation auc = 0.819249
Epoch 90
Validation auc = 0.807042
Epoch 91
Validation auc = 0.818779
Epoch 92
Validation auc = 0.798122
Epoch 93
Validation auc = 0.787324
Epoch 94
Loss = 2.6385e-01, PNorm = 87.0500, GNorm = 6.9214, lr_0 = 8.6198e-04
Validation auc = 0.771362
Epoch 95
Validation auc = 0.778404
Epoch 96
Validation auc = 0.788263
Epoch 97
Validation auc = 0.790141
Epoch 98
Validation auc = 0.800469
Epoch 99
Loss = 1.6231e-01, PNorm = 87.4384, GNorm = 11.7866, lr_0 = 8.5461e-04
Validation auc = 0.807981
Epoch 100
Validation auc = 0.807981
Epoch 101
Validation auc = 0.815023
Epoch 102
Validation auc = 0.811268
Epoch 103
Validation auc = 0.806103
Epoch 104
Loss = 2.0052e-01, PNorm = 87.8214, GNorm = 5.7804, lr_0 = 8.4730e-04
Validation auc = 0.799531
Epoch 105
Validation auc = 0.795305
Epoch 106
Validation auc = 0.784038
Epoch 107
Validation auc = 0.780751
Epoch 108
Validation auc = 0.784977
Epoch 109
Loss = 1.6753e-01, PNorm = 88.2209, GNorm = 7.1587, lr_0 = 8.4006e-04
Validation auc = 0.799531
Epoch 110
Validation auc = 0.806103
Epoch 111
Validation auc = 0.804225
Epoch 112
Validation auc = 0.797183
Epoch 113
Validation auc = 0.800000
Epoch 114
Loss = 1.5130e-01, PNorm = 88.7158, GNorm = 4.7855, lr_0 = 8.3288e-04
Validation auc = 0.808451
Epoch 115
Validation auc = 0.819718
Epoch 116
Validation auc = 0.816901
Epoch 117
Validation auc = 0.791080
Epoch 118
Validation auc = 0.778873
Epoch 119
Loss = 7.1621e-02, PNorm = 89.1463, GNorm = 3.3854, lr_0 = 8.2576e-04
Validation auc = 0.776995
Epoch 120
Validation auc = 0.782160
Epoch 121
Validation auc = 0.788732
Epoch 122
Validation auc = 0.800939
Epoch 123
Validation auc = 0.805634
Epoch 124
Loss = 1.6073e-01, PNorm = 89.5204, GNorm = 5.6746, lr_0 = 8.1870e-04
Validation auc = 0.820657
Epoch 125
Validation auc = 0.830047
Epoch 126
Validation auc = 0.823474
Epoch 127
Validation auc = 0.823474
Epoch 128
Validation auc = 0.813146
Epoch 129
Loss = 1.1554e-01, PNorm = 89.9099, GNorm = 4.1499, lr_0 = 8.1170e-04
Validation auc = 0.815962
Epoch 130
Validation auc = 0.818310
Epoch 131
Validation auc = 0.818310
Epoch 132
Validation auc = 0.818310
Epoch 133
Validation auc = 0.800469
Epoch 134
Loss = 1.3466e-01, PNorm = 90.2709, GNorm = 4.0504, lr_0 = 8.0476e-04
Validation auc = 0.780282
Epoch 135
Validation auc = 0.761972
Epoch 136
Validation auc = 0.754930
Epoch 137
Validation auc = 0.749765
Epoch 138
Validation auc = 0.757746
Epoch 139
Loss = 3.6430e-02, PNorm = 90.6216, GNorm = 0.3394, lr_0 = 7.9788e-04
Validation auc = 0.764319
Epoch 140
Validation auc = 0.773709
Epoch 141
Validation auc = 0.776056
Epoch 142
Validation auc = 0.786385
Epoch 143
Validation auc = 0.789671
Epoch 144
Loss = 8.5666e-02, PNorm = 90.9277, GNorm = 4.0388, lr_0 = 7.9106e-04
Validation auc = 0.795775
Epoch 145
Validation auc = 0.802817
Epoch 146
Validation auc = 0.791080
Epoch 147
Validation auc = 0.790141
Epoch 148
Validation auc = 0.793427
Epoch 149
Loss = 8.0132e-02, PNorm = 91.2297, GNorm = 2.8100, lr_0 = 7.8430e-04
Validation auc = 0.791080
Epoch 150
Validation auc = 0.787324
Epoch 151
Validation auc = 0.784038
Epoch 152
Validation auc = 0.778873
Epoch 153
Validation auc = 0.783568
Epoch 154
Loss = 1.1092e-01, PNorm = 91.5690, GNorm = 6.7524, lr_0 = 7.7759e-04
Validation auc = 0.788732
Epoch 155
Validation auc = 0.793897
Epoch 156
Validation auc = 0.785446
Epoch 157
Validation auc = 0.785446
Epoch 158
Validation auc = 0.799061
Epoch 159
Loss = 5.8438e-02, PNorm = 91.9364, GNorm = 2.1277, lr_0 = 7.7095e-04
Validation auc = 0.809859
Epoch 160
Validation auc = 0.808451
Epoch 161
Validation auc = 0.821127
Epoch 162
Validation auc = 0.823474
Epoch 163
Validation auc = 0.824413
Epoch 164
Loss = 6.2339e-02, PNorm = 92.3380, GNorm = 4.5341, lr_0 = 7.6436e-04
Validation auc = 0.823944
Epoch 165
Validation auc = 0.820188
Epoch 166
Validation auc = 0.807042
Epoch 167
Validation auc = 0.806103
Epoch 168
Validation auc = 0.814085
Epoch 169
Loss = 2.5320e-01, PNorm = 92.7256, GNorm = 6.6883, lr_0 = 7.5782e-04
Validation auc = 0.811737
Epoch 170
Validation auc = 0.807981
Epoch 171
Validation auc = 0.805164
Epoch 172
Validation auc = 0.802347
Epoch 173
Validation auc = 0.800469
Epoch 174
Loss = 1.0112e-01, PNorm = 93.0952, GNorm = 3.6458, lr_0 = 7.5134e-04
Validation auc = 0.801878
Epoch 175
Validation auc = 0.804225
Epoch 176
Validation auc = 0.801878
Epoch 177
Validation auc = 0.799531
Epoch 178
Validation auc = 0.797183
Epoch 179
Loss = 8.4080e-02, PNorm = 93.3931, GNorm = 3.1682, lr_0 = 7.4492e-04
Validation auc = 0.804695
Epoch 180
Validation auc = 0.799531
Epoch 181
Validation auc = 0.794366
Epoch 182
Validation auc = 0.793897
Epoch 183
Validation auc = 0.800000
Epoch 184
Loss = 4.6089e-02, PNorm = 93.6186, GNorm = 5.8461, lr_0 = 7.3855e-04
Validation auc = 0.794366
Epoch 185
Validation auc = 0.794366
Epoch 186
Validation auc = 0.791549
Epoch 187
Validation auc = 0.784507
Epoch 188
Validation auc = 0.785446
Epoch 189
Loss = 4.5552e-02, PNorm = 93.8510, GNorm = 3.3399, lr_0 = 7.3224e-04
Validation auc = 0.784977
Epoch 190
Validation auc = 0.784038
Epoch 191
Validation auc = 0.781221
Epoch 192
Validation auc = 0.784507
Epoch 193
Validation auc = 0.791080
Epoch 194
Loss = 4.0940e-02, PNorm = 94.1242, GNorm = 2.0798, lr_0 = 7.2598e-04
Validation auc = 0.788732
Epoch 195
Validation auc = 0.786385
Epoch 196
Validation auc = 0.785446
Epoch 197
Validation auc = 0.779343
Epoch 198
Validation auc = 0.776995
Epoch 199
Loss = 3.4240e-02, PNorm = 94.4137, GNorm = 0.3873, lr_0 = 7.1977e-04
Validation auc = 0.776995
Epoch 200
Validation auc = 0.783099
Epoch 201
Validation auc = 0.800000
Epoch 202
Validation auc = 0.804695
Epoch 203
Validation auc = 0.806573
Epoch 204
Loss = 1.2606e-02, PNorm = 94.6293, GNorm = 0.2702, lr_0 = 7.1362e-04
Validation auc = 0.807981
Epoch 205
Validation auc = 0.815023
Epoch 206
Validation auc = 0.810329
Epoch 207
Validation auc = 0.804225
Epoch 208
Validation auc = 0.798122
Epoch 209
Loss = 7.0545e-02, PNorm = 94.8410, GNorm = 1.5782, lr_0 = 7.0752e-04
Validation auc = 0.804225
Epoch 210
Validation auc = 0.807981
Epoch 211
Validation auc = 0.799531
Epoch 212
Validation auc = 0.794366
Epoch 213
Validation auc = 0.791549
Epoch 214
Loss = 3.4275e-02, PNorm = 95.0209, GNorm = 2.6128, lr_0 = 7.0147e-04
Validation auc = 0.793427
Epoch 215
Validation auc = 0.791549
Epoch 216
Validation auc = 0.791080
Epoch 217
Validation auc = 0.789202
Epoch 218
Validation auc = 0.792019
Epoch 219
Loss = 1.8996e-02, PNorm = 95.1735, GNorm = 1.1194, lr_0 = 6.9548e-04
Validation auc = 0.791080
Epoch 220
Validation auc = 0.791549
Epoch 221
Validation auc = 0.790610
Epoch 222
Validation auc = 0.793427
Epoch 223
Validation auc = 0.794836
Epoch 224
Loss = 2.6922e-02, PNorm = 95.3345, GNorm = 1.0758, lr_0 = 6.8953e-04
Validation auc = 0.800939
Epoch 225
Validation auc = 0.801878
Epoch 226
Validation auc = 0.803286
Epoch 227
Validation auc = 0.801878
Epoch 228
Validation auc = 0.799531
Epoch 229
Loss = 5.3989e-02, PNorm = 95.4896, GNorm = 4.5449, lr_0 = 6.8364e-04
Validation auc = 0.797183
Epoch 230
Validation auc = 0.795775
Epoch 231
Validation auc = 0.793897
Epoch 232
Validation auc = 0.792019
Epoch 233
Validation auc = 0.789671
Epoch 234
Loss = 1.2440e-02, PNorm = 95.6552, GNorm = 0.3397, lr_0 = 6.7779e-04
Validation auc = 0.791549
Epoch 235
Validation auc = 0.797653
Epoch 236
Validation auc = 0.800469
Epoch 237
Validation auc = 0.799061
Epoch 238
Validation auc = 0.796244
Epoch 239
Loss = 2.5135e-02, PNorm = 95.8613, GNorm = 0.1125, lr_0 = 6.7200e-04
Validation auc = 0.800000
Epoch 240
Validation auc = 0.801878
Epoch 241
Validation auc = 0.792958
Epoch 242
Validation auc = 0.781690
Epoch 243
Validation auc = 0.776995
Epoch 244
Loss = 6.9294e-02, PNorm = 96.1568, GNorm = 4.9384, lr_0 = 6.6625e-04
Validation auc = 0.778404
Epoch 245
Validation auc = 0.784507
Epoch 246
Validation auc = 0.787793
Epoch 247
Validation auc = 0.792019
Epoch 248
Validation auc = 0.790141
Epoch 249
Loss = 1.0838e-02, PNorm = 96.4728, GNorm = 1.4139, lr_0 = 6.6056e-04
Validation auc = 0.784977
Epoch 250
Validation auc = 0.783568
Epoch 251
Validation auc = 0.782629
Epoch 252
Validation auc = 0.780751
Epoch 253
Validation auc = 0.781221
Epoch 254
Loss = 5.5728e-02, PNorm = 96.7172, GNorm = 2.8099, lr_0 = 6.5491e-04
Validation auc = 0.777934
Epoch 255
Validation auc = 0.780751
Epoch 256
Validation auc = 0.788263
Epoch 257
Validation auc = 0.791549
Epoch 258
Validation auc = 0.796714
Epoch 259
Loss = 1.3501e-02, PNorm = 96.9117, GNorm = 0.8225, lr_0 = 6.4931e-04
Validation auc = 0.798122
Epoch 260
Validation auc = 0.797653
Epoch 261
Validation auc = 0.792019
Epoch 262
Validation auc = 0.790610
Epoch 263
Validation auc = 0.795305
Epoch 264
Loss = 4.9870e-03, PNorm = 97.0949, GNorm = 0.1479, lr_0 = 6.4376e-04
Validation auc = 0.795775
Epoch 265
Validation auc = 0.800939
Epoch 266
Validation auc = 0.805164
Epoch 267
Validation auc = 0.806573
Epoch 268
Validation auc = 0.799531
Epoch 269
Loss = 9.3703e-03, PNorm = 97.2409, GNorm = 0.7880, lr_0 = 6.3826e-04
Validation auc = 0.792488
Epoch 270
Validation auc = 0.795305
Epoch 271
Validation auc = 0.788732
Epoch 272
Validation auc = 0.789671
Epoch 273
Validation auc = 0.789202
Epoch 274
Loss = 8.3481e-03, PNorm = 97.3822, GNorm = 0.7766, lr_0 = 6.3280e-04
Validation auc = 0.785446
Epoch 275
Validation auc = 0.784507
Epoch 276
Validation auc = 0.781221
Epoch 277
Validation auc = 0.779812
Epoch 278
Validation auc = 0.786385
Epoch 279
Loss = 2.5571e-03, PNorm = 97.5035, GNorm = 0.5756, lr_0 = 6.2739e-04
Validation auc = 0.790141
Epoch 280
Validation auc = 0.791080
Epoch 281
Validation auc = 0.791549
Epoch 282
Validation auc = 0.791549
Epoch 283
Validation auc = 0.789202
Epoch 284
Loss = 3.6785e-02, PNorm = 97.6053, GNorm = 0.1906, lr_0 = 6.2203e-04
Validation auc = 0.792958
Epoch 285
Validation auc = 0.792488
Epoch 286
Validation auc = 0.792958
Epoch 287
Validation auc = 0.792488
Epoch 288
Validation auc = 0.791549
Epoch 289
Loss = 1.6642e-02, PNorm = 97.7110, GNorm = 2.4236, lr_0 = 6.1671e-04
Validation auc = 0.792958
Epoch 290
Validation auc = 0.798592
Epoch 291
Validation auc = 0.800469
Epoch 292
Validation auc = 0.801878
Epoch 293
Validation auc = 0.803756
Epoch 294
Loss = 1.0998e-02, PNorm = 97.8073, GNorm = 0.4627, lr_0 = 6.1144e-04
Validation auc = 0.803756
Epoch 295
Validation auc = 0.801408
Epoch 296
Validation auc = 0.801878
Epoch 297
Validation auc = 0.802817
Epoch 298
Validation auc = 0.801878
Epoch 299
Loss = 1.6853e-02, PNorm = 97.8895, GNorm = 0.0479, lr_0 = 6.0621e-04
Validation auc = 0.804695
Model 0 best validation auc = 0.830047 on epoch 125
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.775436
Ensemble test auc = 0.775436
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.642254
Epoch 1
Validation auc = 0.701408
Epoch 2
Validation auc = 0.747887
Epoch 3
Validation auc = 0.745070
Epoch 4
Loss = 1.3250e+00, PNorm = 82.4231, GNorm = 14.6966, lr_0 = 6.5000e-04
Validation auc = 0.759624
Epoch 5
Validation auc = 0.776056
Epoch 6
Validation auc = 0.770423
Epoch 7
Validation auc = 0.767606
Epoch 8
Validation auc = 0.781221
Epoch 9
Loss = 1.0334e+00, PNorm = 82.6616, GNorm = 45.7391, lr_0 = 9.9743e-04
Validation auc = 0.798122
Epoch 10
Validation auc = 0.771831
Epoch 11
Validation auc = 0.771831
Epoch 12
Validation auc = 0.756338
Epoch 13
Validation auc = 0.740376
Epoch 14
Loss = 8.0504e-01, PNorm = 82.8789, GNorm = 16.7286, lr_0 = 9.8890e-04
Validation auc = 0.767606
Epoch 15
Validation auc = 0.786385
Epoch 16
Validation auc = 0.773709
Epoch 17
Validation auc = 0.779343
Epoch 18
Validation auc = 0.778873
Epoch 19
Loss = 9.7996e-01, PNorm = 83.0370, GNorm = 31.3252, lr_0 = 9.8045e-04
Validation auc = 0.753052
Epoch 20
Validation auc = 0.689671
Epoch 21
Validation auc = 0.696714
Epoch 22
Validation auc = 0.747887
Epoch 23
Validation auc = 0.767136
Epoch 24
Loss = 7.8344e-01, PNorm = 83.1792, GNorm = 21.2194, lr_0 = 9.7207e-04
Validation auc = 0.786854
Epoch 25
Validation auc = 0.754930
Epoch 26
Validation auc = 0.786854
Epoch 27
Validation auc = 0.723474
Epoch 28
Validation auc = 0.739437
Epoch 29
Loss = 1.1769e+00, PNorm = 83.3152, GNorm = 16.6217, lr_0 = 9.6376e-04
Validation auc = 0.763850
Epoch 30
Validation auc = 0.757277
Epoch 31
Validation auc = 0.773709
Epoch 32
Validation auc = 0.737559
Epoch 33
Validation auc = 0.781221
Epoch 34
Loss = 8.6795e-01, PNorm = 83.4650, GNorm = 23.2544, lr_0 = 9.5552e-04
Validation auc = 0.750704
Epoch 35
Validation auc = 0.685446
Epoch 36
Validation auc = 0.712207
Epoch 37
Validation auc = 0.723944
Epoch 38
Validation auc = 0.743192
Epoch 39
Loss = 1.1547e+00, PNorm = 83.6469, GNorm = 17.2825, lr_0 = 9.4735e-04
Validation auc = 0.727230
Epoch 40
Validation auc = 0.765728
Epoch 41
Validation auc = 0.730516
Epoch 42
Validation auc = 0.744601
Epoch 43
Validation auc = 0.734272
Epoch 44
Loss = 5.3091e-01, PNorm = 83.8757, GNorm = 9.2336, lr_0 = 9.3925e-04
Validation auc = 0.726291
Epoch 45
Validation auc = 0.739906
Epoch 46
Validation auc = 0.753991
Epoch 47
Validation auc = 0.758216
Epoch 48
Validation auc = 0.763850
Epoch 49
Loss = 8.0629e-01, PNorm = 84.1507, GNorm = 31.0370, lr_0 = 9.3122e-04
Validation auc = 0.769953
Epoch 50
Validation auc = 0.764319
Epoch 51
Validation auc = 0.735211
Epoch 52
Validation auc = 0.762441
Epoch 53
Validation auc = 0.724883
Epoch 54
Loss = 3.9274e-01, PNorm = 84.3666, GNorm = 10.5689, lr_0 = 9.2326e-04
Validation auc = 0.741315
Epoch 55
Validation auc = 0.760094
Epoch 56
Validation auc = 0.756338
Epoch 57
Validation auc = 0.755399
Epoch 58
Validation auc = 0.761033
Epoch 59
Loss = 5.7262e-01, PNorm = 84.5786, GNorm = 8.7840, lr_0 = 9.1537e-04
Validation auc = 0.751174
Epoch 60
Validation auc = 0.753991
Epoch 61
Validation auc = 0.753521
Epoch 62
Validation auc = 0.767606
Epoch 63
Validation auc = 0.770892
Epoch 64
Loss = 4.8730e-01, PNorm = 84.8461, GNorm = 14.3292, lr_0 = 9.0754e-04
Validation auc = 0.734742
Epoch 65
Validation auc = 0.743192
Epoch 66
Validation auc = 0.735211
Epoch 67
Validation auc = 0.763850
Epoch 68
Validation auc = 0.784038
Epoch 69
Loss = 4.0450e-01, PNorm = 85.1015, GNorm = 7.7147, lr_0 = 8.9978e-04
Validation auc = 0.794366
Epoch 70
Validation auc = 0.806573
Epoch 71
Validation auc = 0.793897
Epoch 72
Validation auc = 0.800939
Epoch 73
Validation auc = 0.800939
Epoch 74
Loss = 3.2182e-01, PNorm = 85.3579, GNorm = 3.6741, lr_0 = 8.9209e-04
Validation auc = 0.792958
Epoch 75
Validation auc = 0.769484
Epoch 76
Validation auc = 0.736620
Epoch 77
Validation auc = 0.742254
Epoch 78
Validation auc = 0.741315
Epoch 79
Loss = 3.3781e-01, PNorm = 85.6471, GNorm = 4.9919, lr_0 = 8.8447e-04
Validation auc = 0.743192
Epoch 80
Validation auc = 0.744601
Epoch 81
Validation auc = 0.750704
Epoch 82
Validation auc = 0.739906
Epoch 83
Validation auc = 0.740845
Epoch 84
Loss = 3.7075e-01, PNorm = 85.9540, GNorm = 9.7263, lr_0 = 8.7691e-04
Validation auc = 0.765728
Epoch 85
Validation auc = 0.785915
Epoch 86
Validation auc = 0.801878
Epoch 87
Validation auc = 0.796244
Epoch 88
Validation auc = 0.783568
Epoch 89
Loss = 2.8743e-01, PNorm = 86.2657, GNorm = 13.4231, lr_0 = 8.6941e-04
Validation auc = 0.784038
Epoch 90
Validation auc = 0.776056
Epoch 91
Validation auc = 0.769953
Epoch 92
Validation auc = 0.783568
Epoch 93
Validation auc = 0.806573
Epoch 94
Loss = 2.4920e-01, PNorm = 86.5851, GNorm = 6.4897, lr_0 = 8.6198e-04
Validation auc = 0.801408
Epoch 95
Validation auc = 0.805634
Epoch 96
Validation auc = 0.795775
Epoch 97
Validation auc = 0.777934
Epoch 98
Validation auc = 0.756338
Epoch 99
Loss = 1.5067e-01, PNorm = 86.9471, GNorm = 6.1497, lr_0 = 8.5461e-04
Validation auc = 0.743662
Epoch 100
Validation auc = 0.742723
Epoch 101
Validation auc = 0.761502
Epoch 102
Validation auc = 0.774648
Epoch 103
Validation auc = 0.796714
Epoch 104
Loss = 1.3249e-01, PNorm = 87.3037, GNorm = 3.8738, lr_0 = 8.4730e-04
Validation auc = 0.800939
Epoch 105
Validation auc = 0.801878
Epoch 106
Validation auc = 0.797183
Epoch 107
Validation auc = 0.798122
Epoch 108
Validation auc = 0.787324
Epoch 109
Loss = 1.3505e-01, PNorm = 87.6428, GNorm = 4.9104, lr_0 = 8.4006e-04
Validation auc = 0.781221
Epoch 110
Validation auc = 0.787793
Epoch 111
Validation auc = 0.799531
Epoch 112
Validation auc = 0.803286
Epoch 113
Validation auc = 0.800000
Epoch 114
Loss = 1.7272e-01, PNorm = 88.0160, GNorm = 2.7273, lr_0 = 8.3288e-04
Validation auc = 0.797183
Epoch 115
Validation auc = 0.794836
Epoch 116
Validation auc = 0.768075
Epoch 117
Validation auc = 0.766197
Epoch 118
Validation auc = 0.762441
Epoch 119
Loss = 2.5327e-01, PNorm = 88.4021, GNorm = 7.2414, lr_0 = 8.2576e-04
Validation auc = 0.763380
Epoch 120
Validation auc = 0.743662
Epoch 121
Validation auc = 0.757746
Epoch 122
Validation auc = 0.764789
Epoch 123
Validation auc = 0.761972
Epoch 124
Loss = 3.2486e-01, PNorm = 88.7947, GNorm = 8.8298, lr_0 = 8.1870e-04
Validation auc = 0.759155
Epoch 125
Validation auc = 0.766197
Epoch 126
Validation auc = 0.780751
Epoch 127
Validation auc = 0.762911
Epoch 128
Validation auc = 0.808451
Epoch 129
Loss = 3.8085e-01, PNorm = 89.2948, GNorm = 17.0428, lr_0 = 8.1170e-04
Validation auc = 0.793427
Epoch 130
Validation auc = 0.805634
Epoch 131
Validation auc = 0.800469
Epoch 132
Validation auc = 0.792488
Epoch 133
Validation auc = 0.794836
Epoch 134
Loss = 1.7033e-01, PNorm = 89.8197, GNorm = 4.0757, lr_0 = 8.0476e-04
Validation auc = 0.797653
Epoch 135
Validation auc = 0.804225
Epoch 136
Validation auc = 0.810798
Epoch 137
Validation auc = 0.820188
Epoch 138
Validation auc = 0.829108
Epoch 139
Loss = 1.1652e-01, PNorm = 90.2396, GNorm = 6.5514, lr_0 = 7.9788e-04
Validation auc = 0.820657
Epoch 140
Validation auc = 0.825352
Epoch 141
Validation auc = 0.828169
Epoch 142
Validation auc = 0.827230
Epoch 143
Validation auc = 0.825822
Epoch 144
Loss = 1.5663e-01, PNorm = 90.6288, GNorm = 5.3756, lr_0 = 7.9106e-04
Validation auc = 0.819249
Epoch 145
Validation auc = 0.818310
Epoch 146
Validation auc = 0.819718
Epoch 147
Validation auc = 0.826291
Epoch 148
Validation auc = 0.817840
Epoch 149
Loss = 1.0391e-01, PNorm = 90.9898, GNorm = 5.4581, lr_0 = 7.8430e-04
Validation auc = 0.814554
Epoch 150
Validation auc = 0.815962
Epoch 151
Validation auc = 0.806103
Epoch 152
Validation auc = 0.811268
Epoch 153
Validation auc = 0.801408
Epoch 154
Loss = 9.2031e-02, PNorm = 91.3444, GNorm = 3.6859, lr_0 = 7.7759e-04
Validation auc = 0.806573
Epoch 155
Validation auc = 0.799061
Epoch 156
Validation auc = 0.805634
Epoch 157
Validation auc = 0.823944
Epoch 158
Validation auc = 0.822535
Epoch 159
Loss = 1.7427e-01, PNorm = 91.7428, GNorm = 5.0868, lr_0 = 7.7095e-04
Validation auc = 0.801878
Epoch 160
Validation auc = 0.787793
Epoch 161
Validation auc = 0.766197
Epoch 162
Validation auc = 0.746009
Epoch 163
Validation auc = 0.760094
Epoch 164
Loss = 1.2749e-01, PNorm = 92.0721, GNorm = 4.9747, lr_0 = 7.6436e-04
Validation auc = 0.792958
Epoch 165
Validation auc = 0.809390
Epoch 166
Validation auc = 0.810798
Epoch 167
Validation auc = 0.817371
Epoch 168
Validation auc = 0.819718
Epoch 169
Loss = 6.7329e-02, PNorm = 92.4847, GNorm = 3.6621, lr_0 = 7.5782e-04
Validation auc = 0.823005
Epoch 170
Validation auc = 0.813146
Epoch 171
Validation auc = 0.803756
Epoch 172
Validation auc = 0.808451
Epoch 173
Validation auc = 0.803756
Epoch 174
Loss = 1.5589e-01, PNorm = 92.8534, GNorm = 6.2327, lr_0 = 7.5134e-04
Validation auc = 0.797183
Epoch 175
Validation auc = 0.791080
Epoch 176
Validation auc = 0.810798
Epoch 177
Validation auc = 0.825822
Epoch 178
Validation auc = 0.830986
Epoch 179
Loss = 8.3297e-02, PNorm = 93.2425, GNorm = 2.9023, lr_0 = 7.4492e-04
Validation auc = 0.831455
Epoch 180
Validation auc = 0.827230
Epoch 181
Validation auc = 0.827230
Epoch 182
Validation auc = 0.824883
Epoch 183
Validation auc = 0.820657
Epoch 184
Loss = 6.9802e-02, PNorm = 93.5238, GNorm = 3.4897, lr_0 = 7.3855e-04
Validation auc = 0.821127
Epoch 185
Validation auc = 0.817840
Epoch 186
Validation auc = 0.812676
Epoch 187
Validation auc = 0.816432
Epoch 188
Validation auc = 0.811737
Epoch 189
Loss = 3.2069e-02, PNorm = 93.7492, GNorm = 0.5843, lr_0 = 7.3224e-04
Validation auc = 0.803286
Epoch 190
Validation auc = 0.806573
Epoch 191
Validation auc = 0.802817
Epoch 192
Validation auc = 0.807512
Epoch 193
Validation auc = 0.814554
Epoch 194
Loss = 5.8895e-02, PNorm = 93.9986, GNorm = 3.2868, lr_0 = 7.2598e-04
Validation auc = 0.809859
Epoch 195
Validation auc = 0.806103
Epoch 196
Validation auc = 0.803756
Epoch 197
Validation auc = 0.803286
Epoch 198
Validation auc = 0.805634
Epoch 199
Loss = 5.3981e-02, PNorm = 94.2944, GNorm = 4.3229, lr_0 = 7.1977e-04
Validation auc = 0.802817
Epoch 200
Validation auc = 0.790610
Epoch 201
Validation auc = 0.785915
Epoch 202
Validation auc = 0.788732
Epoch 203
Validation auc = 0.792958
Epoch 204
Loss = 3.6053e-02, PNorm = 94.5806, GNorm = 2.3375, lr_0 = 7.1362e-04
Validation auc = 0.802347
Epoch 205
Validation auc = 0.799061
Epoch 206
Validation auc = 0.792958
Epoch 207
Validation auc = 0.784977
Epoch 208
Validation auc = 0.778404
Epoch 209
Loss = 4.3031e-02, PNorm = 94.8904, GNorm = 3.3343, lr_0 = 7.0752e-04
Validation auc = 0.775117
Epoch 210
Validation auc = 0.774178
Epoch 211
Validation auc = 0.769014
Epoch 212
Validation auc = 0.769953
Epoch 213
Validation auc = 0.772770
Epoch 214
Loss = 2.6915e-02, PNorm = 95.1000, GNorm = 1.6058, lr_0 = 7.0147e-04
Validation auc = 0.777465
Epoch 215
Validation auc = 0.775587
Epoch 216
Validation auc = 0.773239
Epoch 217
Validation auc = 0.770892
Epoch 218
Validation auc = 0.770892
Epoch 219
Loss = 6.0486e-02, PNorm = 95.2775, GNorm = 2.4790, lr_0 = 6.9548e-04
Validation auc = 0.788263
Epoch 220
Validation auc = 0.798592
Epoch 221
Validation auc = 0.804225
Epoch 222
Validation auc = 0.801408
Epoch 223
Validation auc = 0.797653
Epoch 224
Loss = 2.4035e-02, PNorm = 95.4344, GNorm = 3.1734, lr_0 = 6.8953e-04
Validation auc = 0.800469
Epoch 225
Validation auc = 0.803756
Epoch 226
Validation auc = 0.800939
Epoch 227
Validation auc = 0.790141
Epoch 228
Validation auc = 0.777465
Epoch 229
Loss = 5.1967e-02, PNorm = 95.6431, GNorm = 2.4485, lr_0 = 6.8364e-04
Validation auc = 0.778404
Epoch 230
Validation auc = 0.783099
Epoch 231
Validation auc = 0.782629
Epoch 232
Validation auc = 0.789671
Epoch 233
Validation auc = 0.797653
Epoch 234
Loss = 1.6398e-02, PNorm = 95.8674, GNorm = 1.1478, lr_0 = 6.7779e-04
Validation auc = 0.800939
Epoch 235
Validation auc = 0.808451
Epoch 236
Validation auc = 0.806103
Epoch 237
Validation auc = 0.807981
Epoch 238
Validation auc = 0.796244
Epoch 239
Loss = 6.5219e-03, PNorm = 96.0883, GNorm = 1.9003, lr_0 = 6.7200e-04
Validation auc = 0.789202
Epoch 240
Validation auc = 0.786854
Epoch 241
Validation auc = 0.791549
Epoch 242
Validation auc = 0.794836
Epoch 243
Validation auc = 0.792488
Epoch 244
Loss = 3.6752e-02, PNorm = 96.2846, GNorm = 0.4585, lr_0 = 6.6625e-04
Validation auc = 0.793897
Epoch 245
Validation auc = 0.794836
Epoch 246
Validation auc = 0.797183
Epoch 247
Validation auc = 0.798122
Epoch 248
Validation auc = 0.792488
Epoch 249
Loss = 1.1562e-02, PNorm = 96.5085, GNorm = 1.2753, lr_0 = 6.6056e-04
Validation auc = 0.793427
Epoch 250
Validation auc = 0.803756
Epoch 251
Validation auc = 0.806103
Epoch 252
Validation auc = 0.801878
Epoch 253
Validation auc = 0.801408
Epoch 254
Loss = 3.7310e-02, PNorm = 96.6995, GNorm = 3.0002, lr_0 = 6.5491e-04
Validation auc = 0.800000
Epoch 255
Validation auc = 0.802817
Epoch 256
Validation auc = 0.801878
Epoch 257
Validation auc = 0.800000
Epoch 258
Validation auc = 0.800469
Epoch 259
Loss = 4.1192e-02, PNorm = 96.8724, GNorm = 4.0716, lr_0 = 6.4931e-04
Validation auc = 0.798592
Epoch 260
Validation auc = 0.787793
Epoch 261
Validation auc = 0.779343
Epoch 262
Validation auc = 0.773709
Epoch 263
Validation auc = 0.776056
Epoch 264
Loss = 6.1011e-03, PNorm = 97.0158, GNorm = 0.4118, lr_0 = 6.4376e-04
Validation auc = 0.779343
Epoch 265
Validation auc = 0.779343
Epoch 266
Validation auc = 0.776056
Epoch 267
Validation auc = 0.773239
Epoch 268
Validation auc = 0.775117
Epoch 269
Loss = 4.3456e-02, PNorm = 97.1717, GNorm = 3.3945, lr_0 = 6.3826e-04
Validation auc = 0.781221
Epoch 270
Validation auc = 0.798592
Epoch 271
Validation auc = 0.807981
Epoch 272
Validation auc = 0.813146
Epoch 273
Validation auc = 0.808451
Epoch 274
Loss = 1.6561e-02, PNorm = 97.3192, GNorm = 0.9039, lr_0 = 6.3280e-04
Validation auc = 0.805634
Epoch 275
Validation auc = 0.798592
Epoch 276
Validation auc = 0.792958
Epoch 277
Validation auc = 0.788263
Epoch 278
Validation auc = 0.786385
Epoch 279
Loss = 2.7727e-02, PNorm = 97.4608, GNorm = 2.2562, lr_0 = 6.2739e-04
Validation auc = 0.784507
Epoch 280
Validation auc = 0.785446
Epoch 281
Validation auc = 0.788732
Epoch 282
Validation auc = 0.794836
Epoch 283
Validation auc = 0.794366
Epoch 284
Loss = 3.1375e-02, PNorm = 97.6135, GNorm = 2.4117, lr_0 = 6.2203e-04
Validation auc = 0.796714
Epoch 285
Validation auc = 0.795775
Epoch 286
Validation auc = 0.794836
Epoch 287
Validation auc = 0.793897
Epoch 288
Validation auc = 0.794366
Epoch 289
Loss = 8.2758e-03, PNorm = 97.7590, GNorm = 0.5424, lr_0 = 6.1671e-04
Validation auc = 0.800000
Epoch 290
Validation auc = 0.802347
Epoch 291
Validation auc = 0.802817
Epoch 292
Validation auc = 0.802347
Epoch 293
Validation auc = 0.807981
Epoch 294
Loss = 5.3175e-03, PNorm = 97.8897, GNorm = 0.2046, lr_0 = 6.1144e-04
Validation auc = 0.812207
Epoch 295
Validation auc = 0.810798
Epoch 296
Validation auc = 0.807512
Epoch 297
Validation auc = 0.805634
Epoch 298
Validation auc = 0.805164
Epoch 299
Loss = 4.8655e-03, PNorm = 97.9918, GNorm = 0.6274, lr_0 = 6.0621e-04
Validation auc = 0.801408
Model 0 best validation auc = 0.831455 on epoch 179
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.802529
Ensemble test auc = 0.802529
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.724413
Epoch 1
Validation auc = 0.741315
Epoch 2
Validation auc = 0.761033
Epoch 3
Validation auc = 0.771362
Epoch 4
Loss = 1.5763e+00, PNorm = 82.4246, GNorm = 48.9542, lr_0 = 6.5000e-04
Validation auc = 0.809155
Epoch 5
Validation auc = 0.837559
Epoch 6
Validation auc = 0.797418
Epoch 7
Validation auc = 0.808920
Epoch 8
Validation auc = 0.776056
Epoch 9
Loss = 1.1945e+00, PNorm = 82.6549, GNorm = 14.0834, lr_0 = 9.9743e-04
Validation auc = 0.770657
Epoch 10
Validation auc = 0.780282
Epoch 11
Validation auc = 0.801878
Epoch 12
Validation auc = 0.832864
Epoch 13
Validation auc = 0.769953
Epoch 14
Loss = 9.0002e-01, PNorm = 82.8731, GNorm = 18.0169, lr_0 = 9.8890e-04
Validation auc = 0.781690
Epoch 15
Validation auc = 0.773239
Epoch 16
Validation auc = 0.747418
Epoch 17
Validation auc = 0.716667
Epoch 18
Validation auc = 0.721127
Epoch 19
Loss = 1.0311e+00, PNorm = 83.0251, GNorm = 21.6855, lr_0 = 9.8045e-04
Validation auc = 0.767136
Epoch 20
Validation auc = 0.753521
Epoch 21
Validation auc = 0.738498
Epoch 22
Validation auc = 0.746479
Epoch 23
Validation auc = 0.746948
Epoch 24
Loss = 8.6407e-01, PNorm = 83.1365, GNorm = 11.3694, lr_0 = 9.7207e-04
Validation auc = 0.779343
Epoch 25
Validation auc = 0.781690
Epoch 26
Validation auc = 0.795775
Epoch 27
Validation auc = 0.798592
Epoch 28
Validation auc = 0.793427
Epoch 29
Loss = 5.3580e-01, PNorm = 83.2491, GNorm = 13.0983, lr_0 = 9.6376e-04
Validation auc = 0.756338
Epoch 30
Validation auc = 0.750704
Epoch 31
Validation auc = 0.721596
Epoch 32
Validation auc = 0.735681
Epoch 33
Validation auc = 0.728638
Epoch 34
Loss = 4.4132e-01, PNorm = 83.3919, GNorm = 5.4747, lr_0 = 9.5552e-04
Validation auc = 0.716432
Epoch 35
Validation auc = 0.738498
Epoch 36
Validation auc = 0.752113
Epoch 37
Validation auc = 0.747418
Epoch 38
Validation auc = 0.756338
Epoch 39
Loss = 5.7581e-01, PNorm = 83.5884, GNorm = 10.1422, lr_0 = 9.4735e-04
Validation auc = 0.765728
Epoch 40
Validation auc = 0.738498
Epoch 41
Validation auc = 0.730986
Epoch 42
Validation auc = 0.720188
Epoch 43
Validation auc = 0.728169
Epoch 44
Loss = 4.3102e-01, PNorm = 83.8142, GNorm = 6.4350, lr_0 = 9.3925e-04
Validation auc = 0.732864
Epoch 45
Validation auc = 0.738498
Epoch 46
Validation auc = 0.746009
Epoch 47
Validation auc = 0.747887
Epoch 48
Validation auc = 0.733333
Epoch 49
Loss = 3.9305e-01, PNorm = 84.0376, GNorm = 14.6751, lr_0 = 9.3122e-04
Validation auc = 0.749296
Epoch 50
Validation auc = 0.759624
Epoch 51
Validation auc = 0.753052
Epoch 52
Validation auc = 0.738498
Epoch 53
Validation auc = 0.763850
Epoch 54
Loss = 6.5864e-01, PNorm = 84.3220, GNorm = 27.3725, lr_0 = 9.2326e-04
Validation auc = 0.774178
Epoch 55
Validation auc = 0.775587
Epoch 56
Validation auc = 0.773709
Epoch 57
Validation auc = 0.784977
Epoch 58
Validation auc = 0.776056
Epoch 59
Loss = 5.3865e-01, PNorm = 84.6358, GNorm = 8.2753, lr_0 = 9.1537e-04
Validation auc = 0.796714
Epoch 60
Validation auc = 0.804695
Epoch 61
Validation auc = 0.824413
Epoch 62
Validation auc = 0.825822
Epoch 63
Validation auc = 0.816901
Epoch 64
Loss = 4.4049e-01, PNorm = 84.9539, GNorm = 7.6279, lr_0 = 9.0754e-04
Validation auc = 0.794836
Epoch 65
Validation auc = 0.789671
Epoch 66
Validation auc = 0.783568
Epoch 67
Validation auc = 0.781690
Epoch 68
Validation auc = 0.779343
Epoch 69
Loss = 3.4865e-01, PNorm = 85.2873, GNorm = 14.5127, lr_0 = 8.9978e-04
Validation auc = 0.768545
Epoch 70
Validation auc = 0.761502
Epoch 71
Validation auc = 0.753521
Epoch 72
Validation auc = 0.756338
Epoch 73
Validation auc = 0.749296
Epoch 74
Loss = 2.8886e-01, PNorm = 85.6314, GNorm = 4.4977, lr_0 = 8.9209e-04
Validation auc = 0.751643
Epoch 75
Validation auc = 0.754460
Epoch 76
Validation auc = 0.757746
Epoch 77
Validation auc = 0.763380
Epoch 78
Validation auc = 0.776995
Epoch 79
Loss = 1.1193e-01, PNorm = 85.9754, GNorm = 4.2081, lr_0 = 8.8447e-04
Validation auc = 0.792958
Epoch 80
Validation auc = 0.793897
Epoch 81
Validation auc = 0.791080
Epoch 82
Validation auc = 0.788732
Epoch 83
Validation auc = 0.773239
Epoch 84
Loss = 2.1191e-01, PNorm = 86.3004, GNorm = 7.5409, lr_0 = 8.7691e-04
Validation auc = 0.727700
Epoch 85
Validation auc = 0.736620
Epoch 86
Validation auc = 0.715962
Epoch 87
Validation auc = 0.751174
Epoch 88
Validation auc = 0.770892
Epoch 89
Loss = 2.8872e-01, PNorm = 86.6802, GNorm = 7.4274, lr_0 = 8.6941e-04
Validation auc = 0.783099
Epoch 90
Validation auc = 0.792019
Epoch 91
Validation auc = 0.805634
Epoch 92
Validation auc = 0.796244
Epoch 93
Validation auc = 0.789671
Epoch 94
Loss = 2.7861e-01, PNorm = 87.0367, GNorm = 5.7559, lr_0 = 8.6198e-04
Validation auc = 0.800469
Epoch 95
Validation auc = 0.793427
Epoch 96
Validation auc = 0.805164
Epoch 97
Validation auc = 0.809390
Epoch 98
Validation auc = 0.803756
Epoch 99
Loss = 2.4539e-01, PNorm = 87.3958, GNorm = 5.8012, lr_0 = 8.5461e-04
Validation auc = 0.802817
Epoch 100
Validation auc = 0.799061
Epoch 101
Validation auc = 0.796244
Epoch 102
Validation auc = 0.788732
Epoch 103
Validation auc = 0.776995
Epoch 104
Loss = 3.0435e-01, PNorm = 87.7900, GNorm = 9.0217, lr_0 = 8.4730e-04
Validation auc = 0.792488
Epoch 105
Validation auc = 0.800469
Epoch 106
Validation auc = 0.799061
Epoch 107
Validation auc = 0.781221
Epoch 108
Validation auc = 0.790141
Epoch 109
Loss = 2.8766e-01, PNorm = 88.1746, GNorm = 11.3685, lr_0 = 8.4006e-04
Validation auc = 0.784977
Epoch 110
Validation auc = 0.788263
Epoch 111
Validation auc = 0.795305
Epoch 112
Validation auc = 0.808451
Epoch 113
Validation auc = 0.817371
Epoch 114
Loss = 2.8560e-01, PNorm = 88.5786, GNorm = 8.5049, lr_0 = 8.3288e-04
Validation auc = 0.804695
Epoch 115
Validation auc = 0.794366
Epoch 116
Validation auc = 0.787793
Epoch 117
Validation auc = 0.783099
Epoch 118
Validation auc = 0.781221
Epoch 119
Loss = 6.5241e-02, PNorm = 88.9441, GNorm = 4.1158, lr_0 = 8.2576e-04
Validation auc = 0.777465
Epoch 120
Validation auc = 0.772300
Epoch 121
Validation auc = 0.791080
Epoch 122
Validation auc = 0.789671
Epoch 123
Validation auc = 0.781221
Epoch 124
Loss = 6.5496e-02, PNorm = 89.2799, GNorm = 1.6276, lr_0 = 8.1870e-04
Validation auc = 0.765728
Epoch 125
Validation auc = 0.767606
Epoch 126
Validation auc = 0.761033
Epoch 127
Validation auc = 0.765258
Epoch 128
Validation auc = 0.760563
Epoch 129
Loss = 4.3749e-02, PNorm = 89.6398, GNorm = 2.7572, lr_0 = 8.1170e-04
Validation auc = 0.768545
Epoch 130
Validation auc = 0.775587
Epoch 131
Validation auc = 0.774648
Epoch 132
Validation auc = 0.780751
Epoch 133
Validation auc = 0.774648
Epoch 134
Loss = 4.4056e-02, PNorm = 89.9789, GNorm = 5.3421, lr_0 = 8.0476e-04
Validation auc = 0.774178
Epoch 135
Validation auc = 0.773709
Epoch 136
Validation auc = 0.768075
Epoch 137
Validation auc = 0.766197
Epoch 138
Validation auc = 0.781690
Epoch 139
Loss = 1.1756e-01, PNorm = 90.3434, GNorm = 4.6520, lr_0 = 7.9788e-04
Validation auc = 0.798122
Epoch 140
Validation auc = 0.810798
Epoch 141
Validation auc = 0.813146
Epoch 142
Validation auc = 0.807512
Epoch 143
Validation auc = 0.805164
Epoch 144
Loss = 6.4404e-02, PNorm = 90.6571, GNorm = 0.9305, lr_0 = 7.9106e-04
Validation auc = 0.804225
Epoch 145
Validation auc = 0.798122
Epoch 146
Validation auc = 0.789202
Epoch 147
Validation auc = 0.790610
Epoch 148
Validation auc = 0.796714
Epoch 149
Loss = 2.1387e-01, PNorm = 90.9392, GNorm = 7.8897, lr_0 = 7.8430e-04
Validation auc = 0.804695
Epoch 150
Validation auc = 0.801408
Epoch 151
Validation auc = 0.791080
Epoch 152
Validation auc = 0.788732
Epoch 153
Validation auc = 0.785446
Epoch 154
Loss = 7.0841e-02, PNorm = 91.2565, GNorm = 3.1932, lr_0 = 7.7759e-04
Validation auc = 0.784507
Epoch 155
Validation auc = 0.783099
Epoch 156
Validation auc = 0.786385
Epoch 157
Validation auc = 0.804225
Epoch 158
Validation auc = 0.809859
Epoch 159
Loss = 8.0488e-02, PNorm = 91.5975, GNorm = 6.1233, lr_0 = 7.7095e-04
Validation auc = 0.801408
Epoch 160
Validation auc = 0.790610
Epoch 161
Validation auc = 0.796714
Epoch 162
Validation auc = 0.813615
Epoch 163
Validation auc = 0.817371
Epoch 164
Loss = 1.1752e-01, PNorm = 91.9589, GNorm = 4.1070, lr_0 = 7.6436e-04
Validation auc = 0.815493
Epoch 165
Validation auc = 0.812207
Epoch 166
Validation auc = 0.817371
Epoch 167
Validation auc = 0.815493
Epoch 168
Validation auc = 0.815023
Epoch 169
Loss = 2.8069e-02, PNorm = 92.3435, GNorm = 1.6821, lr_0 = 7.5782e-04
Validation auc = 0.802347
Epoch 170
Validation auc = 0.810329
Epoch 171
Validation auc = 0.816432
Epoch 172
Validation auc = 0.813146
Epoch 173
Validation auc = 0.819249
Epoch 174
Loss = 7.0912e-02, PNorm = 92.6818, GNorm = 4.8703, lr_0 = 7.5134e-04
Validation auc = 0.819249
Epoch 175
Validation auc = 0.815962
Epoch 176
Validation auc = 0.812676
Epoch 177
Validation auc = 0.807512
Epoch 178
Validation auc = 0.806573
Epoch 179
Loss = 4.9490e-02, PNorm = 92.9702, GNorm = 1.5797, lr_0 = 7.4492e-04
Validation auc = 0.804695
Epoch 180
Validation auc = 0.800000
Epoch 181
Validation auc = 0.796714
Epoch 182
Validation auc = 0.801878
Epoch 183
Validation auc = 0.802347
Epoch 184
Loss = 2.8466e-02, PNorm = 93.2525, GNorm = 1.7077, lr_0 = 7.3855e-04
Validation auc = 0.806103
Epoch 185
Validation auc = 0.809390
Epoch 186
Validation auc = 0.816432
Epoch 187
Validation auc = 0.817840
Epoch 188
Validation auc = 0.813615
Epoch 189
Loss = 2.8018e-02, PNorm = 93.5381, GNorm = 0.8158, lr_0 = 7.3224e-04
Validation auc = 0.796714
Epoch 190
Validation auc = 0.791080
Epoch 191
Validation auc = 0.805164
Epoch 192
Validation auc = 0.805164
Epoch 193
Validation auc = 0.802817
Epoch 194
Loss = 1.1552e-01, PNorm = 93.8545, GNorm = 4.3592, lr_0 = 7.2598e-04
Validation auc = 0.801878
Epoch 195
Validation auc = 0.797653
Epoch 196
Validation auc = 0.798122
Epoch 197
Validation auc = 0.791549
Epoch 198
Validation auc = 0.778873
Epoch 199
Loss = 6.0549e-02, PNorm = 94.1537, GNorm = 1.0486, lr_0 = 7.1977e-04
Validation auc = 0.785446
Epoch 200
Validation auc = 0.786385
Epoch 201
Validation auc = 0.785915
Epoch 202
Validation auc = 0.784038
Epoch 203
Validation auc = 0.778404
Epoch 204
Loss = 2.6291e-02, PNorm = 94.4075, GNorm = 2.3579, lr_0 = 7.1362e-04
Validation auc = 0.769484
Epoch 205
Validation auc = 0.760094
Epoch 206
Validation auc = 0.767606
Epoch 207
Validation auc = 0.767606
Epoch 208
Validation auc = 0.772770
Epoch 209
Loss = 1.3377e-02, PNorm = 94.6912, GNorm = 0.9611, lr_0 = 7.0752e-04
Validation auc = 0.768545
Epoch 210
Validation auc = 0.768545
Epoch 211
Validation auc = 0.771831
Epoch 212
Validation auc = 0.775117
Epoch 213
Validation auc = 0.784038
Epoch 214
Loss = 2.3607e-02, PNorm = 94.9251, GNorm = 2.0935, lr_0 = 7.0147e-04
Validation auc = 0.790141
Epoch 215
Validation auc = 0.781221
Epoch 216
Validation auc = 0.783568
Epoch 217
Validation auc = 0.789671
Epoch 218
Validation auc = 0.789671
Epoch 219
Loss = 3.4943e-02, PNorm = 95.1451, GNorm = 0.5697, lr_0 = 6.9548e-04
Validation auc = 0.796244
Epoch 220
Validation auc = 0.795305
Epoch 221
Validation auc = 0.796714
Epoch 222
Validation auc = 0.798122
Epoch 223
Validation auc = 0.799061
Epoch 224
Loss = 1.4808e-02, PNorm = 95.3428, GNorm = 2.1712, lr_0 = 6.8953e-04
Validation auc = 0.792958
Epoch 225
Validation auc = 0.790610
Epoch 226
Validation auc = 0.792958
Epoch 227
Validation auc = 0.800469
Epoch 228
Validation auc = 0.800469
Epoch 229
Loss = 1.6935e-02, PNorm = 95.5081, GNorm = 1.2655, lr_0 = 6.8364e-04
Validation auc = 0.801878
Epoch 230
Validation auc = 0.800469
Epoch 231
Validation auc = 0.807042
Epoch 232
Validation auc = 0.807981
Epoch 233
Validation auc = 0.808920
Epoch 234
Loss = 3.3385e-02, PNorm = 95.6621, GNorm = 1.8998, lr_0 = 6.7779e-04
Validation auc = 0.810798
Epoch 235
Validation auc = 0.807512
Epoch 236
Validation auc = 0.803756
Epoch 237
Validation auc = 0.801878
Epoch 238
Validation auc = 0.801878
Epoch 239
Loss = 5.8801e-02, PNorm = 95.8180, GNorm = 1.7769, lr_0 = 6.7200e-04
Validation auc = 0.798122
Epoch 240
Validation auc = 0.797183
Epoch 241
Validation auc = 0.796714
Epoch 242
Validation auc = 0.797653
Epoch 243
Validation auc = 0.802817
Epoch 244
Loss = 7.6229e-02, PNorm = 95.9921, GNorm = 1.7135, lr_0 = 6.6625e-04
Validation auc = 0.807042
Epoch 245
Validation auc = 0.810798
Epoch 246
Validation auc = 0.806573
Epoch 247
Validation auc = 0.805164
Epoch 248
Validation auc = 0.800469
Epoch 249
Loss = 4.5524e-02, PNorm = 96.2284, GNorm = 3.7335, lr_0 = 6.6056e-04
Validation auc = 0.803286
Epoch 250
Validation auc = 0.806573
Epoch 251
Validation auc = 0.808451
Epoch 252
Validation auc = 0.810798
Epoch 253
Validation auc = 0.802347
Epoch 254
Loss = 5.9015e-02, PNorm = 96.4363, GNorm = 6.6628, lr_0 = 6.5491e-04
Validation auc = 0.804225
Epoch 255
Validation auc = 0.807981
Epoch 256
Validation auc = 0.807981
Epoch 257
Validation auc = 0.807042
Epoch 258
Validation auc = 0.806573
Epoch 259
Loss = 7.6248e-03, PNorm = 96.6054, GNorm = 0.5396, lr_0 = 6.4931e-04
Validation auc = 0.807981
Epoch 260
Validation auc = 0.801408
Epoch 261
Validation auc = 0.801408
Epoch 262
Validation auc = 0.794836
Epoch 263
Validation auc = 0.796714
Epoch 264
Loss = 1.7977e-02, PNorm = 96.7511, GNorm = 0.1282, lr_0 = 6.4376e-04
Validation auc = 0.800000
Epoch 265
Validation auc = 0.800469
Epoch 266
Validation auc = 0.804695
Epoch 267
Validation auc = 0.800469
Epoch 268
Validation auc = 0.806103
Epoch 269
Loss = 1.4566e-02, PNorm = 96.9396, GNorm = 0.3044, lr_0 = 6.3826e-04
Validation auc = 0.799531
Epoch 270
Validation auc = 0.803521
Epoch 271
Validation auc = 0.806103
Epoch 272
Validation auc = 0.807042
Epoch 273
Validation auc = 0.810798
Epoch 274
Loss = 5.9948e-02, PNorm = 97.1422, GNorm = 3.4919, lr_0 = 6.3280e-04
Validation auc = 0.807981
Epoch 275
Validation auc = 0.800469
Epoch 276
Validation auc = 0.796714
Epoch 277
Validation auc = 0.787324
Epoch 278
Validation auc = 0.784507
Epoch 279
Loss = 1.3237e-02, PNorm = 97.3418, GNorm = 2.0001, lr_0 = 6.2739e-04
Validation auc = 0.783099
Epoch 280
Validation auc = 0.786385
Epoch 281
Validation auc = 0.790141
Epoch 282
Validation auc = 0.793897
Epoch 283
Validation auc = 0.798357
Epoch 284
Loss = 8.3877e-03, PNorm = 97.4670, GNorm = 0.8335, lr_0 = 6.2203e-04
Validation auc = 0.800000
Epoch 285
Validation auc = 0.797653
Epoch 286
Validation auc = 0.795775
Epoch 287
Validation auc = 0.796244
Epoch 288
Validation auc = 0.794366
Epoch 289
Loss = 2.1655e-03, PNorm = 97.5537, GNorm = 0.0966, lr_0 = 6.1671e-04
Validation auc = 0.796714
Epoch 290
Validation auc = 0.800000
Epoch 291
Validation auc = 0.803286
Epoch 292
Validation auc = 0.807981
Epoch 293
Validation auc = 0.808451
Epoch 294
Loss = 2.0953e-02, PNorm = 97.6437, GNorm = 1.0390, lr_0 = 6.1144e-04
Validation auc = 0.807042
Epoch 295
Validation auc = 0.806573
Epoch 296
Validation auc = 0.809859
Epoch 297
Validation auc = 0.807512
Epoch 298
Validation auc = 0.808920
Epoch 299
Loss = 4.5800e-03, PNorm = 97.7073, GNorm = 0.7105, lr_0 = 6.0621e-04
Validation auc = 0.808451
Model 0 best validation auc = 0.837559 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.860325
Ensemble test auc = 0.860325
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.708920
Epoch 1
Validation auc = 0.730047
Epoch 2
Validation auc = 0.758685
Epoch 3
Validation auc = 0.760094
Epoch 4
Loss = 1.3735e+00, PNorm = 82.4260, GNorm = 31.7557, lr_0 = 6.5000e-04
Validation auc = 0.799296
Epoch 5
Validation auc = 0.817371
Epoch 6
Validation auc = 0.824413
Epoch 7
Validation auc = 0.815962
Epoch 8
Validation auc = 0.799061
Epoch 9
Loss = 1.0791e+00, PNorm = 82.6693, GNorm = 16.5024, lr_0 = 9.9743e-04
Validation auc = 0.810563
Epoch 10
Validation auc = 0.825352
Epoch 11
Validation auc = 0.826056
Epoch 12
Validation auc = 0.820657
Epoch 13
Validation auc = 0.780282
Epoch 14
Loss = 9.3156e-01, PNorm = 82.8941, GNorm = 32.1199, lr_0 = 9.8890e-04
Validation auc = 0.756808
Epoch 15
Validation auc = 0.753052
Epoch 16
Validation auc = 0.699531
Epoch 17
Validation auc = 0.784977
Epoch 18
Validation auc = 0.779812
Epoch 19
Loss = 1.0392e+00, PNorm = 83.0423, GNorm = 31.9784, lr_0 = 9.8045e-04
Validation auc = 0.768545
Epoch 20
Validation auc = 0.774178
Epoch 21
Validation auc = 0.739437
Epoch 22
Validation auc = 0.711268
Epoch 23
Validation auc = 0.756808
Epoch 24
Loss = 7.1049e-01, PNorm = 83.1560, GNorm = 11.7769, lr_0 = 9.7207e-04
Validation auc = 0.780282
Epoch 25
Validation auc = 0.790141
Epoch 26
Validation auc = 0.792958
Epoch 27
Validation auc = 0.774648
Epoch 28
Validation auc = 0.748826
Epoch 29
Loss = 5.3709e-01, PNorm = 83.2686, GNorm = 22.4362, lr_0 = 9.6376e-04
Validation auc = 0.754460
Epoch 30
Validation auc = 0.738498
Epoch 31
Validation auc = 0.748357
Epoch 32
Validation auc = 0.718310
Epoch 33
Validation auc = 0.705164
Epoch 34
Loss = 6.3083e-01, PNorm = 83.3854, GNorm = 12.1253, lr_0 = 9.5552e-04
Validation auc = 0.723944
Epoch 35
Validation auc = 0.730516
Epoch 36
Validation auc = 0.740845
Epoch 37
Validation auc = 0.757746
Epoch 38
Validation auc = 0.760563
Epoch 39
Loss = 6.5680e-01, PNorm = 83.5438, GNorm = 9.6708, lr_0 = 9.4735e-04
Validation auc = 0.761502
Epoch 40
Validation auc = 0.765728
Epoch 41
Validation auc = 0.744601
Epoch 42
Validation auc = 0.742723
Epoch 43
Validation auc = 0.758216
Epoch 44
Loss = 3.5322e-01, PNorm = 83.7393, GNorm = 7.3462, lr_0 = 9.3925e-04
Validation auc = 0.769953
Epoch 45
Validation auc = 0.776995
Epoch 46
Validation auc = 0.773239
Epoch 47
Validation auc = 0.769953
Epoch 48
Validation auc = 0.766667
Epoch 49
Loss = 3.7868e-01, PNorm = 84.0190, GNorm = 13.9485, lr_0 = 9.3122e-04
Validation auc = 0.728169
Epoch 50
Validation auc = 0.743662
Epoch 51
Validation auc = 0.738498
Epoch 52
Validation auc = 0.765258
Epoch 53
Validation auc = 0.757277
Epoch 54
Loss = 4.8987e-01, PNorm = 84.3737, GNorm = 20.5593, lr_0 = 9.2326e-04
Validation auc = 0.776526
Epoch 55
Validation auc = 0.774648
Epoch 56
Validation auc = 0.781221
Epoch 57
Validation auc = 0.776526
Epoch 58
Validation auc = 0.767606
Epoch 59
Loss = 6.6353e-01, PNorm = 84.7599, GNorm = 16.3872, lr_0 = 9.1537e-04
Validation auc = 0.786385
Epoch 60
Validation auc = 0.783099
Epoch 61
Validation auc = 0.773239
Epoch 62
Validation auc = 0.776526
Epoch 63
Validation auc = 0.786385
Epoch 64
Loss = 4.1389e-01, PNorm = 85.0980, GNorm = 9.6745, lr_0 = 9.0754e-04
Validation auc = 0.782629
Epoch 65
Validation auc = 0.777465
Epoch 66
Validation auc = 0.772770
Epoch 67
Validation auc = 0.776526
Epoch 68
Validation auc = 0.794366
Epoch 69
Loss = 4.0205e-01, PNorm = 85.4424, GNorm = 17.9548, lr_0 = 8.9978e-04
Validation auc = 0.792488
Epoch 70
Validation auc = 0.790141
Epoch 71
Validation auc = 0.805634
Epoch 72
Validation auc = 0.779812
Epoch 73
Validation auc = 0.763380
Epoch 74
Loss = 2.3886e-01, PNorm = 85.7902, GNorm = 4.9222, lr_0 = 8.9209e-04
Validation auc = 0.747887
Epoch 75
Validation auc = 0.761972
Epoch 76
Validation auc = 0.759624
Epoch 77
Validation auc = 0.759155
Epoch 78
Validation auc = 0.769484
Epoch 79
Loss = 1.4391e-01, PNorm = 86.1748, GNorm = 3.7589, lr_0 = 8.8447e-04
Validation auc = 0.777465
Epoch 80
Validation auc = 0.798122
Epoch 81
Validation auc = 0.814085
Epoch 82
Validation auc = 0.815962
Epoch 83
Validation auc = 0.795775
Epoch 84
Loss = 2.4155e-01, PNorm = 86.5321, GNorm = 10.8736, lr_0 = 8.7691e-04
Validation auc = 0.759155
Epoch 85
Validation auc = 0.760094
Epoch 86
Validation auc = 0.771362
Epoch 87
Validation auc = 0.766197
Epoch 88
Validation auc = 0.756808
Epoch 89
Loss = 3.4749e-01, PNorm = 86.9270, GNorm = 6.4483, lr_0 = 8.6941e-04
Validation auc = 0.782629
Epoch 90
Validation auc = 0.786854
Epoch 91
Validation auc = 0.780282
Epoch 92
Validation auc = 0.779343
Epoch 93
Validation auc = 0.792958
Epoch 94
Loss = 1.2349e-01, PNorm = 87.3557, GNorm = 3.8296, lr_0 = 8.6198e-04
Validation auc = 0.792019
Epoch 95
Validation auc = 0.794366
Epoch 96
Validation auc = 0.799061
Epoch 97
Validation auc = 0.758216
Epoch 98
Validation auc = 0.763850
Epoch 99
Loss = 1.9497e-01, PNorm = 87.8060, GNorm = 4.1834, lr_0 = 8.5461e-04
Validation auc = 0.792019
Epoch 100
Validation auc = 0.774648
Epoch 101
Validation auc = 0.757277
Epoch 102
Validation auc = 0.766667
Epoch 103
Validation auc = 0.768545
Epoch 104
Loss = 1.5613e-01, PNorm = 88.2053, GNorm = 4.5441, lr_0 = 8.4730e-04
Validation auc = 0.776526
Epoch 105
Validation auc = 0.774178
Epoch 106
Validation auc = 0.776526
Epoch 107
Validation auc = 0.777465
Epoch 108
Validation auc = 0.788263
Epoch 109
Loss = 1.5897e-01, PNorm = 88.6152, GNorm = 6.3145, lr_0 = 8.4006e-04
Validation auc = 0.797653
Epoch 110
Validation auc = 0.807042
Epoch 111
Validation auc = 0.788732
Epoch 112
Validation auc = 0.795775
Epoch 113
Validation auc = 0.800469
Epoch 114
Loss = 1.3419e-01, PNorm = 88.9876, GNorm = 8.0528, lr_0 = 8.3288e-04
Validation auc = 0.800000
Epoch 115
Validation auc = 0.795305
Epoch 116
Validation auc = 0.796714
Epoch 117
Validation auc = 0.801408
Epoch 118
Validation auc = 0.795305
Epoch 119
Loss = 7.1080e-02, PNorm = 89.3891, GNorm = 4.3013, lr_0 = 8.2576e-04
Validation auc = 0.787793
Epoch 120
Validation auc = 0.787793
Epoch 121
Validation auc = 0.806573
Epoch 122
Validation auc = 0.808451
Epoch 123
Validation auc = 0.807042
Epoch 124
Loss = 1.9609e-01, PNorm = 89.7886, GNorm = 6.9141, lr_0 = 8.1870e-04
Validation auc = 0.807512
Epoch 125
Validation auc = 0.803756
Epoch 126
Validation auc = 0.795305
Epoch 127
Validation auc = 0.788263
Epoch 128
Validation auc = 0.782160
Epoch 129
Loss = 1.6228e-01, PNorm = 90.1713, GNorm = 6.0400, lr_0 = 8.1170e-04
Validation auc = 0.788732
Epoch 130
Validation auc = 0.797183
Epoch 131
Validation auc = 0.810329
Epoch 132
Validation auc = 0.817371
Epoch 133
Validation auc = 0.813146
Epoch 134
Loss = 3.8205e-02, PNorm = 90.5413, GNorm = 0.8624, lr_0 = 8.0476e-04
Validation auc = 0.805164
Epoch 135
Validation auc = 0.802817
Epoch 136
Validation auc = 0.801408
Epoch 137
Validation auc = 0.807981
Epoch 138
Validation auc = 0.800939
Epoch 139
Loss = 1.5851e-01, PNorm = 90.8840, GNorm = 7.8436, lr_0 = 7.9788e-04
Validation auc = 0.802817
Epoch 140
Validation auc = 0.803286
Epoch 141
Validation auc = 0.800469
Epoch 142
Validation auc = 0.803756
Epoch 143
Validation auc = 0.802817
Epoch 144
Loss = 1.1410e-01, PNorm = 91.2568, GNorm = 3.3347, lr_0 = 7.9106e-04
Validation auc = 0.797183
Epoch 145
Validation auc = 0.789202
Epoch 146
Validation auc = 0.781690
Epoch 147
Validation auc = 0.784038
Epoch 148
Validation auc = 0.789671
Epoch 149
Loss = 4.4834e-02, PNorm = 91.6220, GNorm = 1.7501, lr_0 = 7.8430e-04
Validation auc = 0.785915
Epoch 150
Validation auc = 0.783568
Epoch 151
Validation auc = 0.776056
Epoch 152
Validation auc = 0.771831
Epoch 153
Validation auc = 0.772300
Epoch 154
Loss = 5.5402e-02, PNorm = 91.9551, GNorm = 3.1111, lr_0 = 7.7759e-04
Validation auc = 0.776056
Epoch 155
Validation auc = 0.776056
Epoch 156
Validation auc = 0.782160
Epoch 157
Validation auc = 0.777934
Epoch 158
Validation auc = 0.767136
Epoch 159
Loss = 1.5847e-01, PNorm = 92.2656, GNorm = 8.3232, lr_0 = 7.7095e-04
Validation auc = 0.776056
Epoch 160
Validation auc = 0.787324
Epoch 161
Validation auc = 0.790141
Epoch 162
Validation auc = 0.792488
Epoch 163
Validation auc = 0.781690
Epoch 164
Loss = 3.8556e-02, PNorm = 92.6048, GNorm = 2.5921, lr_0 = 7.6436e-04
Validation auc = 0.769484
Epoch 165
Validation auc = 0.776056
Epoch 166
Validation auc = 0.772300
Epoch 167
Validation auc = 0.769953
Epoch 168
Validation auc = 0.762441
Epoch 169
Loss = 4.6947e-02, PNorm = 92.9143, GNorm = 2.7441, lr_0 = 7.5782e-04
Validation auc = 0.758685
Epoch 170
Validation auc = 0.773239
Epoch 171
Validation auc = 0.782160
Epoch 172
Validation auc = 0.790141
Epoch 173
Validation auc = 0.795775
Epoch 174
Loss = 4.6639e-02, PNorm = 93.2637, GNorm = 2.9573, lr_0 = 7.5134e-04
Validation auc = 0.803286
Epoch 175
Validation auc = 0.796244
Epoch 176
Validation auc = 0.796714
Epoch 177
Validation auc = 0.796714
Epoch 178
Validation auc = 0.794836
Epoch 179
Loss = 3.0882e-02, PNorm = 93.6059, GNorm = 0.6352, lr_0 = 7.4492e-04
Validation auc = 0.789671
Epoch 180
Validation auc = 0.782629
Epoch 181
Validation auc = 0.788263
Epoch 182
Validation auc = 0.791549
Epoch 183
Validation auc = 0.784977
Epoch 184
Loss = 2.1401e-02, PNorm = 93.9043, GNorm = 1.7014, lr_0 = 7.3855e-04
Validation auc = 0.787793
Epoch 185
Validation auc = 0.777465
Epoch 186
Validation auc = 0.781221
Epoch 187
Validation auc = 0.790141
Epoch 188
Validation auc = 0.791549
Epoch 189
Loss = 1.5626e-02, PNorm = 94.1284, GNorm = 1.6234, lr_0 = 7.3224e-04
Validation auc = 0.792488
Epoch 190
Validation auc = 0.795305
Epoch 191
Validation auc = 0.800000
Epoch 192
Validation auc = 0.802817
Epoch 193
Validation auc = 0.803286
Epoch 194
Loss = 1.0031e-02, PNorm = 94.3009, GNorm = 0.9488, lr_0 = 7.2598e-04
Validation auc = 0.804225
Epoch 195
Validation auc = 0.801878
Epoch 196
Validation auc = 0.802817
Epoch 197
Validation auc = 0.798122
Epoch 198
Validation auc = 0.796244
Epoch 199
Loss = 3.6680e-02, PNorm = 94.4550, GNorm = 1.6770, lr_0 = 7.1977e-04
Validation auc = 0.793427
Epoch 200
Validation auc = 0.788732
Epoch 201
Validation auc = 0.780751
Epoch 202
Validation auc = 0.785915
Epoch 203
Validation auc = 0.787324
Epoch 204
Loss = 2.7149e-02, PNorm = 94.6085, GNorm = 2.5534, lr_0 = 7.1362e-04
Validation auc = 0.792019
Epoch 205
Validation auc = 0.795305
Epoch 206
Validation auc = 0.796714
Epoch 207
Validation auc = 0.798122
Epoch 208
Validation auc = 0.802817
Epoch 209
Loss = 1.4820e-02, PNorm = 94.7553, GNorm = 1.6092, lr_0 = 7.0752e-04
Validation auc = 0.804695
Epoch 210
Validation auc = 0.799061
Epoch 211
Validation auc = 0.799531
Epoch 212
Validation auc = 0.801408
Epoch 213
Validation auc = 0.791080
Epoch 214
Loss = 3.0031e-02, PNorm = 94.9425, GNorm = 1.2810, lr_0 = 7.0147e-04
Validation auc = 0.784977
Epoch 215
Validation auc = 0.776995
Epoch 216
Validation auc = 0.777934
Epoch 217
Validation auc = 0.790610
Epoch 218
Validation auc = 0.791549
Epoch 219
Loss = 1.0531e-02, PNorm = 95.1501, GNorm = 0.6591, lr_0 = 6.9548e-04
Validation auc = 0.783568
Epoch 220
Validation auc = 0.781690
Epoch 221
Validation auc = 0.781690
Epoch 222
Validation auc = 0.787324
Epoch 223
Validation auc = 0.786854
Epoch 224
Loss = 3.0641e-02, PNorm = 95.3712, GNorm = 4.2352, lr_0 = 6.8953e-04
Validation auc = 0.782160
Epoch 225
Validation auc = 0.776526
Epoch 226
Validation auc = 0.773239
Epoch 227
Validation auc = 0.771831
Epoch 228
Validation auc = 0.777465
Epoch 229
Loss = 3.5635e-02, PNorm = 95.5930, GNorm = 3.4721, lr_0 = 6.8364e-04
Validation auc = 0.780282
Epoch 230
Validation auc = 0.781221
Epoch 231
Validation auc = 0.784038
Epoch 232
Validation auc = 0.786854
Epoch 233
Validation auc = 0.788263
Epoch 234
Loss = 4.8501e-02, PNorm = 95.7731, GNorm = 1.6418, lr_0 = 6.7779e-04
Validation auc = 0.785915
Epoch 235
Validation auc = 0.785915
Epoch 236
Validation auc = 0.783099
Epoch 237
Validation auc = 0.781221
Epoch 238
Validation auc = 0.779343
Epoch 239
Loss = 2.2877e-02, PNorm = 95.9448, GNorm = 0.8016, lr_0 = 6.7200e-04
Validation auc = 0.782629
Epoch 240
Validation auc = 0.784038
Epoch 241
Validation auc = 0.785446
Epoch 242
Validation auc = 0.781221
Epoch 243
Validation auc = 0.779343
Epoch 244
Loss = 4.6775e-03, PNorm = 96.1068, GNorm = 0.0867, lr_0 = 6.6625e-04
Validation auc = 0.775117
Epoch 245
Validation auc = 0.775117
Epoch 246
Validation auc = 0.781221
Epoch 247
Validation auc = 0.786854
Epoch 248
Validation auc = 0.785446
Epoch 249
Loss = 7.7462e-03, PNorm = 96.2565, GNorm = 0.8257, lr_0 = 6.6056e-04
Validation auc = 0.787324
Epoch 250
Validation auc = 0.785915
Epoch 251
Validation auc = 0.787324
Epoch 252
Validation auc = 0.790141
Epoch 253
Validation auc = 0.785446
Epoch 254
Loss = 5.5072e-02, PNorm = 96.4478, GNorm = 1.6231, lr_0 = 6.5491e-04
Validation auc = 0.778404
Epoch 255
Validation auc = 0.777465
Epoch 256
Validation auc = 0.780751
Epoch 257
Validation auc = 0.780751
Epoch 258
Validation auc = 0.779343
Epoch 259
Loss = 1.4531e-02, PNorm = 96.6627, GNorm = 2.3073, lr_0 = 6.4931e-04
Validation auc = 0.776995
Epoch 260
Validation auc = 0.781690
Epoch 261
Validation auc = 0.795775
Epoch 262
Validation auc = 0.800469
Epoch 263
Validation auc = 0.802347
Epoch 264
Loss = 9.2129e-02, PNorm = 96.8728, GNorm = 1.9589, lr_0 = 6.4376e-04
Validation auc = 0.802347
Epoch 265
Validation auc = 0.800000
Epoch 266
Validation auc = 0.799531
Epoch 267
Validation auc = 0.799531
Epoch 268
Validation auc = 0.796714
Epoch 269
Loss = 1.6912e-02, PNorm = 97.0973, GNorm = 0.7208, lr_0 = 6.3826e-04
Validation auc = 0.796244
Epoch 270
Validation auc = 0.796714
Epoch 271
Validation auc = 0.798122
Epoch 272
Validation auc = 0.798122
Epoch 273
Validation auc = 0.787793
Epoch 274
Loss = 4.7971e-02, PNorm = 97.2966, GNorm = 2.9605, lr_0 = 6.3280e-04
Validation auc = 0.780282
Epoch 275
Validation auc = 0.786854
Epoch 276
Validation auc = 0.786385
Epoch 277
Validation auc = 0.773239
Epoch 278
Validation auc = 0.788732
Epoch 279
Loss = 1.3318e-01, PNorm = 97.6736, GNorm = 9.0137, lr_0 = 6.2739e-04
Validation auc = 0.796244
Epoch 280
Validation auc = 0.794836
Epoch 281
Validation auc = 0.787559
Epoch 282
Validation auc = 0.798592
Epoch 283
Validation auc = 0.815962
Epoch 284
Loss = 6.3298e-02, PNorm = 98.1868, GNorm = 6.9625, lr_0 = 6.2203e-04
Validation auc = 0.825822
Epoch 285
Validation auc = 0.817371
Epoch 286
Validation auc = 0.807512
Epoch 287
Validation auc = 0.801408
Epoch 288
Validation auc = 0.801878
Epoch 289
Loss = 9.5262e-02, PNorm = 98.6671, GNorm = 2.0082, lr_0 = 6.1671e-04
Validation auc = 0.799531
Epoch 290
Validation auc = 0.795775
Epoch 291
Validation auc = 0.791549
Epoch 292
Validation auc = 0.788263
Epoch 293
Validation auc = 0.784507
Epoch 294
Loss = 5.0729e-02, PNorm = 99.0265, GNorm = 2.9453, lr_0 = 6.1144e-04
Validation auc = 0.786854
Epoch 295
Validation auc = 0.789671
Epoch 296
Validation auc = 0.790141
Epoch 297
Validation auc = 0.791549
Epoch 298
Validation auc = 0.788263
Epoch 299
Loss = 1.0977e-02, PNorm = 99.2696, GNorm = 0.3288, lr_0 = 6.0621e-04
Validation auc = 0.791549
Model 0 best validation auc = 0.826056 on epoch 11
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.878387
Ensemble test auc = 0.878387
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.662911
Epoch 1
Validation auc = 0.753991
Epoch 2
Validation auc = 0.679812
Epoch 3
Validation auc = 0.785446
Epoch 4
Loss = 1.5816e+00, PNorm = 82.4256, GNorm = 48.8321, lr_0 = 6.5000e-04
Validation auc = 0.835211
Epoch 5
Validation auc = 0.839906
Epoch 6
Validation auc = 0.753991
Epoch 7
Validation auc = 0.816901
Epoch 8
Validation auc = 0.812676
Epoch 9
Loss = 1.3732e+00, PNorm = 82.6582, GNorm = 61.2897, lr_0 = 9.9743e-04
Validation auc = 0.801878
Epoch 10
Validation auc = 0.803286
Epoch 11
Validation auc = 0.784038
Epoch 12
Validation auc = 0.780751
Epoch 13
Validation auc = 0.738967
Epoch 14
Loss = 1.1436e+00, PNorm = 82.8794, GNorm = 10.8639, lr_0 = 9.8890e-04
Validation auc = 0.733803
Epoch 15
Validation auc = 0.719249
Epoch 16
Validation auc = 0.718779
Epoch 17
Validation auc = 0.723474
Epoch 18
Validation auc = 0.733803
Epoch 19
Loss = 6.5437e-01, PNorm = 83.0225, GNorm = 18.8438, lr_0 = 9.8045e-04
Validation auc = 0.778873
Epoch 20
Validation auc = 0.773709
Epoch 21
Validation auc = 0.784507
Epoch 22
Validation auc = 0.781690
Epoch 23
Validation auc = 0.738028
Epoch 24
Loss = 6.7508e-01, PNorm = 83.1286, GNorm = 15.8552, lr_0 = 9.7207e-04
Validation auc = 0.728169
Epoch 25
Validation auc = 0.739906
Epoch 26
Validation auc = 0.765258
Epoch 27
Validation auc = 0.765728
Epoch 28
Validation auc = 0.761972
Epoch 29
Loss = 5.2651e-01, PNorm = 83.2511, GNorm = 26.3022, lr_0 = 9.6376e-04
Validation auc = 0.738498
Epoch 30
Validation auc = 0.743662
Epoch 31
Validation auc = 0.728638
Epoch 32
Validation auc = 0.717840
Epoch 33
Validation auc = 0.758685
Epoch 34
Loss = 9.3426e-01, PNorm = 83.4157, GNorm = 13.6624, lr_0 = 9.5552e-04
Validation auc = 0.734742
Epoch 35
Validation auc = 0.730516
Epoch 36
Validation auc = 0.746009
Epoch 37
Validation auc = 0.773239
Epoch 38
Validation auc = 0.761972
Epoch 39
Loss = 4.9785e-01, PNorm = 83.6156, GNorm = 11.2607, lr_0 = 9.4735e-04
Validation auc = 0.772770
Epoch 40
Validation auc = 0.757746
Epoch 41
Validation auc = 0.743662
Epoch 42
Validation auc = 0.737559
Epoch 43
Validation auc = 0.731925
Epoch 44
Loss = 5.8667e-01, PNorm = 83.9019, GNorm = 15.7586, lr_0 = 9.3925e-04
Validation auc = 0.775587
Epoch 45
Validation auc = 0.809859
Epoch 46
Validation auc = 0.799531
Epoch 47
Validation auc = 0.807981
Epoch 48
Validation auc = 0.780751
Epoch 49
Loss = 3.6124e-01, PNorm = 84.2049, GNorm = 5.4037, lr_0 = 9.3122e-04
Validation auc = 0.740376
Epoch 50
Validation auc = 0.742723
Epoch 51
Validation auc = 0.748826
Epoch 52
Validation auc = 0.785446
Epoch 53
Validation auc = 0.818310
Epoch 54
Loss = 4.3388e-01, PNorm = 84.4808, GNorm = 9.9840, lr_0 = 9.2326e-04
Validation auc = 0.818310
Epoch 55
Validation auc = 0.816432
Epoch 56
Validation auc = 0.813146
Epoch 57
Validation auc = 0.806573
Epoch 58
Validation auc = 0.797183
Epoch 59
Loss = 1.0150e+00, PNorm = 84.7773, GNorm = 37.6220, lr_0 = 9.1537e-04
Validation auc = 0.771831
Epoch 60
Validation auc = 0.754460
Epoch 61
Validation auc = 0.780751
Epoch 62
Validation auc = 0.794836
Epoch 63
Validation auc = 0.798122
Epoch 64
Loss = 5.0723e-01, PNorm = 85.0540, GNorm = 13.6766, lr_0 = 9.0754e-04
Validation auc = 0.789671
Epoch 65
Validation auc = 0.775587
Epoch 66
Validation auc = 0.762441
Epoch 67
Validation auc = 0.753991
Epoch 68
Validation auc = 0.787793
Epoch 69
Loss = 3.8815e-01, PNorm = 85.3507, GNorm = 10.7931, lr_0 = 8.9978e-04
Validation auc = 0.809390
Epoch 70
Validation auc = 0.837089
Epoch 71
Validation auc = 0.835211
Epoch 72
Validation auc = 0.816901
Epoch 73
Validation auc = 0.801878
Epoch 74
Loss = 2.2828e-01, PNorm = 85.6486, GNorm = 9.1366, lr_0 = 8.9209e-04
Validation auc = 0.802347
Epoch 75
Validation auc = 0.810329
Epoch 76
Validation auc = 0.819718
Epoch 77
Validation auc = 0.810329
Epoch 78
Validation auc = 0.792019
Epoch 79
Loss = 2.9527e-01, PNorm = 85.9512, GNorm = 14.4852, lr_0 = 8.8447e-04
Validation auc = 0.808920
Epoch 80
Validation auc = 0.800000
Epoch 81
Validation auc = 0.774648
Epoch 82
Validation auc = 0.770892
Epoch 83
Validation auc = 0.788263
Epoch 84
Loss = 2.1339e-01, PNorm = 86.2344, GNorm = 6.3290, lr_0 = 8.7691e-04
Validation auc = 0.792488
Epoch 85
Validation auc = 0.790141
Epoch 86
Validation auc = 0.786854
Epoch 87
Validation auc = 0.786854
Epoch 88
Validation auc = 0.779812
Epoch 89
Loss = 1.5725e-01, PNorm = 86.5628, GNorm = 4.5894, lr_0 = 8.6941e-04
Validation auc = 0.786385
Epoch 90
Validation auc = 0.788732
Epoch 91
Validation auc = 0.784977
Epoch 92
Validation auc = 0.793427
Epoch 93
Validation auc = 0.812676
Epoch 94
Loss = 2.1041e-01, PNorm = 86.9445, GNorm = 6.9982, lr_0 = 8.6198e-04
Validation auc = 0.830047
Epoch 95
Validation auc = 0.823944
Epoch 96
Validation auc = 0.818779
Epoch 97
Validation auc = 0.799531
Epoch 98
Validation auc = 0.780751
Epoch 99
Loss = 1.6234e-01, PNorm = 87.3886, GNorm = 7.0934, lr_0 = 8.5461e-04
Validation auc = 0.740845
Epoch 100
Validation auc = 0.719718
Epoch 101
Validation auc = 0.721127
Epoch 102
Validation auc = 0.739437
Epoch 103
Validation auc = 0.775587
Epoch 104
Loss = 1.3197e-01, PNorm = 87.7932, GNorm = 4.8252, lr_0 = 8.4730e-04
Validation auc = 0.792958
Epoch 105
Validation auc = 0.790610
Epoch 106
Validation auc = 0.783099
Epoch 107
Validation auc = 0.788263
Epoch 108
Validation auc = 0.761502
Epoch 109
Loss = 2.7336e-01, PNorm = 88.2103, GNorm = 7.6092, lr_0 = 8.4006e-04
Validation auc = 0.773709
Epoch 110
Validation auc = 0.775117
Epoch 111
Validation auc = 0.780282
Epoch 112
Validation auc = 0.762911
Epoch 113
Validation auc = 0.753521
Epoch 114
Loss = 1.3444e-01, PNorm = 88.6553, GNorm = 4.3733, lr_0 = 8.3288e-04
Validation auc = 0.761033
Epoch 115
Validation auc = 0.768075
Epoch 116
Validation auc = 0.771831
Epoch 117
Validation auc = 0.788732
Epoch 118
Validation auc = 0.788732
Epoch 119
Loss = 1.4298e-01, PNorm = 89.0811, GNorm = 3.3620, lr_0 = 8.2576e-04
Validation auc = 0.784507
Epoch 120
Validation auc = 0.779343
Epoch 121
Validation auc = 0.775117
Epoch 122
Validation auc = 0.763850
Epoch 123
Validation auc = 0.763380
Epoch 124
Loss = 1.1467e-01, PNorm = 89.4526, GNorm = 1.5715, lr_0 = 8.1870e-04
Validation auc = 0.764789
Epoch 125
Validation auc = 0.768545
Epoch 126
Validation auc = 0.779343
Epoch 127
Validation auc = 0.786385
Epoch 128
Validation auc = 0.795775
Epoch 129
Loss = 1.5305e-01, PNorm = 89.8227, GNorm = 2.1118, lr_0 = 8.1170e-04
Validation auc = 0.804695
Epoch 130
Validation auc = 0.808920
Epoch 131
Validation auc = 0.806573
Epoch 132
Validation auc = 0.801878
Epoch 133
Validation auc = 0.808920
Epoch 134
Loss = 1.6005e-01, PNorm = 90.2301, GNorm = 7.9586, lr_0 = 8.0476e-04
Validation auc = 0.800000
Epoch 135
Validation auc = 0.797183
Epoch 136
Validation auc = 0.788732
Epoch 137
Validation auc = 0.795775
Epoch 138
Validation auc = 0.799061
Epoch 139
Loss = 1.0271e-01, PNorm = 90.6154, GNorm = 2.6071, lr_0 = 7.9788e-04
Validation auc = 0.795775
Epoch 140
Validation auc = 0.810329
Epoch 141
Validation auc = 0.816432
Epoch 142
Validation auc = 0.800469
Epoch 143
Validation auc = 0.780751
Epoch 144
Loss = 2.0197e-01, PNorm = 90.9628, GNorm = 6.3826, lr_0 = 7.9106e-04
Validation auc = 0.777934
Epoch 145
Validation auc = 0.761033
Epoch 146
Validation auc = 0.760094
Epoch 147
Validation auc = 0.757746
Epoch 148
Validation auc = 0.765728
Epoch 149
Loss = 7.3017e-02, PNorm = 91.3248, GNorm = 0.9975, lr_0 = 7.8430e-04
Validation auc = 0.762911
Epoch 150
Validation auc = 0.764789
Epoch 151
Validation auc = 0.773709
Epoch 152
Validation auc = 0.776526
Epoch 153
Validation auc = 0.782160
Epoch 154
Loss = 6.7900e-02, PNorm = 91.6388, GNorm = 1.5587, lr_0 = 7.7759e-04
Validation auc = 0.784977
Epoch 155
Validation auc = 0.785915
Epoch 156
Validation auc = 0.786385
Epoch 157
Validation auc = 0.785915
Epoch 158
Validation auc = 0.792958
Epoch 159
Loss = 1.5728e-01, PNorm = 91.9578, GNorm = 9.4978, lr_0 = 7.7095e-04
Validation auc = 0.797183
Epoch 160
Validation auc = 0.800000
Epoch 161
Validation auc = 0.789202
Epoch 162
Validation auc = 0.779812
Epoch 163
Validation auc = 0.780282
Epoch 164
Loss = 1.2892e-01, PNorm = 92.3031, GNorm = 7.2113, lr_0 = 7.6436e-04
Validation auc = 0.777465
Epoch 165
Validation auc = 0.777934
Epoch 166
Validation auc = 0.783568
Epoch 167
Validation auc = 0.781690
Epoch 168
Validation auc = 0.782160
Epoch 169
Loss = 6.7017e-02, PNorm = 92.6058, GNorm = 2.2734, lr_0 = 7.5782e-04
Validation auc = 0.782629
Epoch 170
Validation auc = 0.782160
Epoch 171
Validation auc = 0.783568
Epoch 172
Validation auc = 0.776056
Epoch 173
Validation auc = 0.764319
Epoch 174
Loss = 5.9964e-02, PNorm = 92.9344, GNorm = 4.2181, lr_0 = 7.5134e-04
Validation auc = 0.769484
Epoch 175
Validation auc = 0.781221
Epoch 176
Validation auc = 0.784977
Epoch 177
Validation auc = 0.788263
Epoch 178
Validation auc = 0.785446
Epoch 179
Loss = 1.0443e-01, PNorm = 93.3337, GNorm = 2.5741, lr_0 = 7.4492e-04
Validation auc = 0.783099
Epoch 180
Validation auc = 0.778404
Epoch 181
Validation auc = 0.768545
Epoch 182
Validation auc = 0.764789
Epoch 183
Validation auc = 0.757746
Epoch 184
Loss = 1.1664e-01, PNorm = 93.7162, GNorm = 7.6887, lr_0 = 7.3855e-04
Validation auc = 0.752582
Epoch 185
Validation auc = 0.761502
Epoch 186
Validation auc = 0.762911
Epoch 187
Validation auc = 0.763850
Epoch 188
Validation auc = 0.778404
Epoch 189
Loss = 2.4088e-02, PNorm = 94.1073, GNorm = 1.5630, lr_0 = 7.3224e-04
Validation auc = 0.784977
Epoch 190
Validation auc = 0.787324
Epoch 191
Validation auc = 0.789202
Epoch 192
Validation auc = 0.782629
Epoch 193
Validation auc = 0.779343
Epoch 194
Loss = 3.0703e-02, PNorm = 94.4370, GNorm = 1.5650, lr_0 = 7.2598e-04
Validation auc = 0.774648
Epoch 195
Validation auc = 0.770892
Epoch 196
Validation auc = 0.766197
Epoch 197
Validation auc = 0.768545
Epoch 198
Validation auc = 0.776056
Epoch 199
Loss = 3.7547e-02, PNorm = 94.7074, GNorm = 2.1850, lr_0 = 7.1977e-04
Validation auc = 0.769953
Epoch 200
Validation auc = 0.771831
Epoch 201
Validation auc = 0.762441
Epoch 202
Validation auc = 0.766197
Epoch 203
Validation auc = 0.774178
Epoch 204
Loss = 1.4490e-01, PNorm = 95.0303, GNorm = 8.0190, lr_0 = 7.1362e-04
Validation auc = 0.784507
Epoch 205
Validation auc = 0.783099
Epoch 206
Validation auc = 0.784977
Epoch 207
Validation auc = 0.788732
Epoch 208
Validation auc = 0.794836
Epoch 209
Loss = 3.8468e-02, PNorm = 95.3223, GNorm = 1.3287, lr_0 = 7.0752e-04
Validation auc = 0.793427
Epoch 210
Validation auc = 0.792019
Epoch 211
Validation auc = 0.787324
Epoch 212
Validation auc = 0.791549
Epoch 213
Validation auc = 0.798122
Epoch 214
Loss = 1.3192e-01, PNorm = 95.5939, GNorm = 4.5209, lr_0 = 7.0147e-04
Validation auc = 0.802347
Epoch 215
Validation auc = 0.800939
Epoch 216
Validation auc = 0.800000
Epoch 217
Validation auc = 0.798122
Epoch 218
Validation auc = 0.799061
Epoch 219
Loss = 3.6791e-02, PNorm = 95.8679, GNorm = 0.5110, lr_0 = 6.9548e-04
Validation auc = 0.796244
Epoch 220
Validation auc = 0.794836
Epoch 221
Validation auc = 0.790610
Epoch 222
Validation auc = 0.784507
Epoch 223
Validation auc = 0.785446
Epoch 224
Loss = 8.7921e-02, PNorm = 96.0869, GNorm = 5.1874, lr_0 = 6.8953e-04
Validation auc = 0.785446
Epoch 225
Validation auc = 0.791080
Epoch 226
Validation auc = 0.790141
Epoch 227
Validation auc = 0.791080
Epoch 228
Validation auc = 0.790141
Epoch 229
Loss = 6.1401e-02, PNorm = 96.2339, GNorm = 3.3053, lr_0 = 6.8364e-04
Validation auc = 0.784507
Epoch 230
Validation auc = 0.782629
Epoch 231
Validation auc = 0.786854
Epoch 232
Validation auc = 0.784507
Epoch 233
Validation auc = 0.776526
Epoch 234
Loss = 1.6687e-02, PNorm = 96.3483, GNorm = 0.2391, lr_0 = 6.7779e-04
Validation auc = 0.778404
Epoch 235
Validation auc = 0.778873
Epoch 236
Validation auc = 0.775117
Epoch 237
Validation auc = 0.776056
Epoch 238
Validation auc = 0.779343
Epoch 239
Loss = 2.0842e-02, PNorm = 96.4800, GNorm = 0.4631, lr_0 = 6.7200e-04
Validation auc = 0.778404
Epoch 240
Validation auc = 0.777934
Epoch 241
Validation auc = 0.783568
Epoch 242
Validation auc = 0.784038
Epoch 243
Validation auc = 0.782629
Epoch 244
Loss = 1.5482e-02, PNorm = 96.6608, GNorm = 1.3668, lr_0 = 6.6625e-04
Validation auc = 0.782160
Epoch 245
Validation auc = 0.788732
Epoch 246
Validation auc = 0.793897
Epoch 247
Validation auc = 0.794836
Epoch 248
Validation auc = 0.796244
Epoch 249
Loss = 1.1489e-02, PNorm = 96.8200, GNorm = 1.0835, lr_0 = 6.6056e-04
Validation auc = 0.792958
Epoch 250
Validation auc = 0.789671
Epoch 251
Validation auc = 0.785446
Epoch 252
Validation auc = 0.785446
Epoch 253
Validation auc = 0.782629
Epoch 254
Loss = 1.2774e-02, PNorm = 96.9653, GNorm = 1.8261, lr_0 = 6.5491e-04
Validation auc = 0.781221
Epoch 255
Validation auc = 0.776995
Epoch 256
Validation auc = 0.775587
Epoch 257
Validation auc = 0.776995
Epoch 258
Validation auc = 0.776526
Epoch 259
Loss = 1.0162e-02, PNorm = 97.0962, GNorm = 0.1921, lr_0 = 6.4931e-04
Validation auc = 0.779812
Epoch 260
Validation auc = 0.777934
Epoch 261
Validation auc = 0.780751
Epoch 262
Validation auc = 0.786385
Epoch 263
Validation auc = 0.793427
Epoch 264
Loss = 7.4491e-03, PNorm = 97.2144, GNorm = 0.9485, lr_0 = 6.4376e-04
Validation auc = 0.793427
Epoch 265
Validation auc = 0.792958
Epoch 266
Validation auc = 0.792488
Epoch 267
Validation auc = 0.794366
Epoch 268
Validation auc = 0.794366
Epoch 269
Loss = 1.2012e-02, PNorm = 97.3009, GNorm = 0.1554, lr_0 = 6.3826e-04
Validation auc = 0.794366
Epoch 270
Validation auc = 0.797653
Epoch 271
Validation auc = 0.797183
Epoch 272
Validation auc = 0.793427
Epoch 273
Validation auc = 0.791080
Epoch 274
Loss = 1.0301e-02, PNorm = 97.3756, GNorm = 0.2981, lr_0 = 6.3280e-04
Validation auc = 0.789202
Epoch 275
Validation auc = 0.786854
Epoch 276
Validation auc = 0.779812
Epoch 277
Validation auc = 0.778404
Epoch 278
Validation auc = 0.773239
Epoch 279
Loss = 2.3020e-02, PNorm = 97.4623, GNorm = 2.6352, lr_0 = 6.2739e-04
Validation auc = 0.767606
Epoch 280
Validation auc = 0.769484
Epoch 281
Validation auc = 0.764789
Epoch 282
Validation auc = 0.760094
Epoch 283
Validation auc = 0.761033
Epoch 284
Loss = 1.9737e-02, PNorm = 97.5950, GNorm = 1.0523, lr_0 = 6.2203e-04
Validation auc = 0.765258
Epoch 285
Validation auc = 0.770892
Epoch 286
Validation auc = 0.787793
Epoch 287
Validation auc = 0.789671
Epoch 288
Validation auc = 0.785446
Epoch 289
Loss = 1.0006e-02, PNorm = 97.7436, GNorm = 0.4105, lr_0 = 6.1671e-04
Validation auc = 0.783568
Epoch 290
Validation auc = 0.783099
Epoch 291
Validation auc = 0.779343
Epoch 292
Validation auc = 0.776995
Epoch 293
Validation auc = 0.774178
Epoch 294
Loss = 1.1323e-02, PNorm = 97.8760, GNorm = 0.7119, lr_0 = 6.1144e-04
Validation auc = 0.771362
Epoch 295
Validation auc = 0.763380
Epoch 296
Validation auc = 0.759624
Epoch 297
Validation auc = 0.756808
Epoch 298
Validation auc = 0.752582
Epoch 299
Loss = 1.9393e-02, PNorm = 97.9891, GNorm = 2.8950, lr_0 = 6.0621e-04
Validation auc = 0.753521
Model 0 best validation auc = 0.839906 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.864539
Ensemble test auc = 0.864539
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.683099
Epoch 1
Validation auc = 0.737559
Epoch 2
Validation auc = 0.777465
Epoch 3
Validation auc = 0.804225
Epoch 4
Loss = 1.5178e+00, PNorm = 82.4321, GNorm = 21.3205, lr_0 = 6.5000e-04
Validation auc = 0.832864
Epoch 5
Validation auc = 0.766667
Epoch 6
Validation auc = 0.793897
Epoch 7
Validation auc = 0.800469
Epoch 8
Validation auc = 0.789202
Epoch 9
Loss = 9.9727e-01, PNorm = 82.6729, GNorm = 19.0189, lr_0 = 9.9743e-04
Validation auc = 0.780751
Epoch 10
Validation auc = 0.757746
Epoch 11
Validation auc = 0.768075
Epoch 12
Validation auc = 0.766667
Epoch 13
Validation auc = 0.763850
Epoch 14
Loss = 1.0972e+00, PNorm = 82.9232, GNorm = 37.9173, lr_0 = 9.8890e-04
Validation auc = 0.761502
Epoch 15
Validation auc = 0.744601
Epoch 16
Validation auc = 0.753991
Epoch 17
Validation auc = 0.741784
Epoch 18
Validation auc = 0.750235
Epoch 19
Loss = 1.0058e+00, PNorm = 83.0971, GNorm = 27.9053, lr_0 = 9.8045e-04
Validation auc = 0.767136
Epoch 20
Validation auc = 0.780751
Epoch 21
Validation auc = 0.782160
Epoch 22
Validation auc = 0.748826
Epoch 23
Validation auc = 0.696714
Epoch 24
Loss = 9.7023e-01, PNorm = 83.2395, GNorm = 11.5409, lr_0 = 9.7207e-04
Validation auc = 0.716901
Epoch 25
Validation auc = 0.725822
Epoch 26
Validation auc = 0.774178
Epoch 27
Validation auc = 0.781221
Epoch 28
Validation auc = 0.774648
Epoch 29
Loss = 7.7637e-01, PNorm = 83.3830, GNorm = 28.6570, lr_0 = 9.6376e-04
Validation auc = 0.760563
Epoch 30
Validation auc = 0.748826
Epoch 31
Validation auc = 0.734272
Epoch 32
Validation auc = 0.769953
Epoch 33
Validation auc = 0.795305
Epoch 34
Loss = 4.7383e-01, PNorm = 83.5662, GNorm = 9.1418, lr_0 = 9.5552e-04
Validation auc = 0.769953
Epoch 35
Validation auc = 0.778873
Epoch 36
Validation auc = 0.750235
Epoch 37
Validation auc = 0.722066
Epoch 38
Validation auc = 0.722066
Epoch 39
Loss = 8.4305e-01, PNorm = 83.7830, GNorm = 24.5118, lr_0 = 9.4735e-04
Validation auc = 0.744131
Epoch 40
Validation auc = 0.760094
Epoch 41
Validation auc = 0.769484
Epoch 42
Validation auc = 0.778404
Epoch 43
Validation auc = 0.772300
Epoch 44
Loss = 5.8921e-01, PNorm = 84.0435, GNorm = 7.6385, lr_0 = 9.3925e-04
Validation auc = 0.762911
Epoch 45
Validation auc = 0.756808
Epoch 46
Validation auc = 0.745070
Epoch 47
Validation auc = 0.745540
Epoch 48
Validation auc = 0.753521
Epoch 49
Loss = 4.7444e-01, PNorm = 84.3351, GNorm = 6.8597, lr_0 = 9.3122e-04
Validation auc = 0.769014
Epoch 50
Validation auc = 0.790610
Epoch 51
Validation auc = 0.793897
Epoch 52
Validation auc = 0.793897
Epoch 53
Validation auc = 0.775587
Epoch 54
Loss = 3.8995e-01, PNorm = 84.6460, GNorm = 10.7269, lr_0 = 9.2326e-04
Validation auc = 0.784507
Epoch 55
Validation auc = 0.774178
Epoch 56
Validation auc = 0.771831
Epoch 57
Validation auc = 0.765258
Epoch 58
Validation auc = 0.769014
Epoch 59
Loss = 4.0552e-01, PNorm = 84.9108, GNorm = 8.0656, lr_0 = 9.1537e-04
Validation auc = 0.769014
Epoch 60
Validation auc = 0.761972
Epoch 61
Validation auc = 0.767136
Epoch 62
Validation auc = 0.787324
Epoch 63
Validation auc = 0.776056
Epoch 64
Loss = 1.7025e-01, PNorm = 85.2555, GNorm = 4.0618, lr_0 = 9.0754e-04
Validation auc = 0.786854
Epoch 65
Validation auc = 0.789202
Epoch 66
Validation auc = 0.788263
Epoch 67
Validation auc = 0.775117
Epoch 68
Validation auc = 0.790141
Epoch 69
Loss = 3.7819e-01, PNorm = 85.5573, GNorm = 10.2828, lr_0 = 8.9978e-04
Validation auc = 0.786854
Epoch 70
Validation auc = 0.794366
Epoch 71
Validation auc = 0.783099
Epoch 72
Validation auc = 0.770892
Epoch 73
Validation auc = 0.785446
Epoch 74
Loss = 1.7468e-01, PNorm = 85.8747, GNorm = 6.8583, lr_0 = 8.9209e-04
Validation auc = 0.800939
Epoch 75
Validation auc = 0.797653
Epoch 76
Validation auc = 0.797653
Epoch 77
Validation auc = 0.795775
Epoch 78
Validation auc = 0.771831
Epoch 79
Loss = 2.4549e-01, PNorm = 86.2230, GNorm = 8.2238, lr_0 = 8.8447e-04
Validation auc = 0.770423
Epoch 80
Validation auc = 0.803286
Epoch 81
Validation auc = 0.805634
Epoch 82
Validation auc = 0.795305
Epoch 83
Validation auc = 0.777465
Epoch 84
Loss = 1.1755e-01, PNorm = 86.5487, GNorm = 5.4560, lr_0 = 8.7691e-04
Validation auc = 0.763380
Epoch 85
Validation auc = 0.768545
Epoch 86
Validation auc = 0.778404
Epoch 87
Validation auc = 0.791549
Epoch 88
Validation auc = 0.802817
Epoch 89
Loss = 2.1460e-01, PNorm = 86.8717, GNorm = 3.6288, lr_0 = 8.6941e-04
Validation auc = 0.814554
Epoch 90
Validation auc = 0.817371
Epoch 91
Validation auc = 0.815493
Epoch 92
Validation auc = 0.811268
Epoch 93
Validation auc = 0.815493
Epoch 94
Loss = 2.9539e-01, PNorm = 87.2359, GNorm = 11.2774, lr_0 = 8.6198e-04
Validation auc = 0.797183
Epoch 95
Validation auc = 0.780282
Epoch 96
Validation auc = 0.772300
Epoch 97
Validation auc = 0.774178
Epoch 98
Validation auc = 0.772300
Epoch 99
Loss = 2.1184e-01, PNorm = 87.6273, GNorm = 4.8146, lr_0 = 8.5461e-04
Validation auc = 0.768545
Epoch 100
Validation auc = 0.782160
Epoch 101
Validation auc = 0.789202
Epoch 102
Validation auc = 0.798592
Epoch 103
Validation auc = 0.798122
Epoch 104
Loss = 1.9147e-01, PNorm = 88.0521, GNorm = 10.3517, lr_0 = 8.4730e-04
Validation auc = 0.802817
Epoch 105
Validation auc = 0.806103
Epoch 106
Validation auc = 0.803286
Epoch 107
Validation auc = 0.794366
Epoch 108
Validation auc = 0.776995
Epoch 109
Loss = 2.3731e-01, PNorm = 88.4687, GNorm = 5.6739, lr_0 = 8.4006e-04
Validation auc = 0.757746
Epoch 110
Validation auc = 0.752582
Epoch 111
Validation auc = 0.772770
Epoch 112
Validation auc = 0.790610
Epoch 113
Validation auc = 0.812207
Epoch 114
Loss = 2.0978e-01, PNorm = 88.9644, GNorm = 7.4599, lr_0 = 8.3288e-04
Validation auc = 0.816901
Epoch 115
Validation auc = 0.823944
Epoch 116
Validation auc = 0.825352
Epoch 117
Validation auc = 0.827230
Epoch 118
Validation auc = 0.821127
Epoch 119
Loss = 1.0565e-01, PNorm = 89.4284, GNorm = 3.9149, lr_0 = 8.2576e-04
Validation auc = 0.813146
Epoch 120
Validation auc = 0.810329
Epoch 121
Validation auc = 0.797183
Epoch 122
Validation auc = 0.786385
Epoch 123
Validation auc = 0.782160
Epoch 124
Loss = 2.2147e-01, PNorm = 89.8647, GNorm = 7.7982, lr_0 = 8.1870e-04
Validation auc = 0.798122
Epoch 125
Validation auc = 0.807981
Epoch 126
Validation auc = 0.807042
Epoch 127
Validation auc = 0.798592
Epoch 128
Validation auc = 0.776056
Epoch 129
Loss = 9.7875e-02, PNorm = 90.3035, GNorm = 7.0710, lr_0 = 8.1170e-04
Validation auc = 0.759155
Epoch 130
Validation auc = 0.775587
Epoch 131
Validation auc = 0.780751
Epoch 132
Validation auc = 0.782160
Epoch 133
Validation auc = 0.774178
Epoch 134
Loss = 5.9136e-02, PNorm = 90.7651, GNorm = 2.1985, lr_0 = 8.0476e-04
Validation auc = 0.767136
Epoch 135
Validation auc = 0.778873
Epoch 136
Validation auc = 0.778404
Epoch 137
Validation auc = 0.780751
Epoch 138
Validation auc = 0.794366
Epoch 139
Loss = 6.4006e-02, PNorm = 91.1746, GNorm = 5.3712, lr_0 = 7.9788e-04
Validation auc = 0.804695
Epoch 140
Validation auc = 0.806573
Epoch 141
Validation auc = 0.817840
Epoch 142
Validation auc = 0.825822
Epoch 143
Validation auc = 0.827700
Epoch 144
Loss = 5.7400e-02, PNorm = 91.5739, GNorm = 2.6679, lr_0 = 7.9106e-04
Validation auc = 0.824413
Epoch 145
Validation auc = 0.807981
Epoch 146
Validation auc = 0.789202
Epoch 147
Validation auc = 0.780282
Epoch 148
Validation auc = 0.765258
Epoch 149
Loss = 1.1463e-01, PNorm = 92.0015, GNorm = 3.5732, lr_0 = 7.8430e-04
Validation auc = 0.759155
Epoch 150
Validation auc = 0.777934
Epoch 151
Validation auc = 0.784507
Epoch 152
Validation auc = 0.788263
Epoch 153
Validation auc = 0.789202
Epoch 154
Loss = 1.4998e-01, PNorm = 92.4334, GNorm = 3.6150, lr_0 = 7.7759e-04
Validation auc = 0.800000
Epoch 155
Validation auc = 0.810329
Epoch 156
Validation auc = 0.815023
Epoch 157
Validation auc = 0.808451
Epoch 158
Validation auc = 0.806103
Epoch 159
Loss = 9.0257e-02, PNorm = 92.8037, GNorm = 3.1374, lr_0 = 7.7095e-04
Validation auc = 0.802817
Epoch 160
Validation auc = 0.793897
Epoch 161
Validation auc = 0.789671
Epoch 162
Validation auc = 0.782160
Epoch 163
Validation auc = 0.781690
Epoch 164
Loss = 8.1897e-02, PNorm = 93.1745, GNorm = 5.6606, lr_0 = 7.6436e-04
Validation auc = 0.789671
Epoch 165
Validation auc = 0.792958
Epoch 166
Validation auc = 0.803286
Epoch 167
Validation auc = 0.801408
Epoch 168
Validation auc = 0.799531
Epoch 169
Loss = 4.7757e-02, PNorm = 93.5650, GNorm = 1.2479, lr_0 = 7.5782e-04
Validation auc = 0.794366
Epoch 170
Validation auc = 0.795775
Epoch 171
Validation auc = 0.785446
Epoch 172
Validation auc = 0.783568
Epoch 173
Validation auc = 0.805634
Epoch 174
Loss = 7.2447e-02, PNorm = 93.9862, GNorm = 1.3372, lr_0 = 7.5134e-04
Validation auc = 0.810329
Epoch 175
Validation auc = 0.797183
Epoch 176
Validation auc = 0.793897
Epoch 177
Validation auc = 0.790610
Epoch 178
Validation auc = 0.787324
Epoch 179
Loss = 3.8957e-02, PNorm = 94.3851, GNorm = 2.8700, lr_0 = 7.4492e-04
Validation auc = 0.778404
Epoch 180
Validation auc = 0.779812
Epoch 181
Validation auc = 0.785915
Epoch 182
Validation auc = 0.786854
Epoch 183
Validation auc = 0.785915
Epoch 184
Loss = 1.1103e-02, PNorm = 94.7748, GNorm = 0.4103, lr_0 = 7.3855e-04
Validation auc = 0.784038
Epoch 185
Validation auc = 0.781690
Epoch 186
Validation auc = 0.777934
Epoch 187
Validation auc = 0.762441
Epoch 188
Validation auc = 0.749765
Epoch 189
Loss = 2.0384e-01, PNorm = 95.0934, GNorm = 11.9001, lr_0 = 7.3224e-04
Validation auc = 0.766197
Epoch 190
Validation auc = 0.782629
Epoch 191
Validation auc = 0.782160
Epoch 192
Validation auc = 0.776526
Epoch 193
Validation auc = 0.779812
Epoch 194
Loss = 7.5288e-02, PNorm = 95.4910, GNorm = 1.3050, lr_0 = 7.2598e-04
Validation auc = 0.783099
Epoch 195
Validation auc = 0.797653
Epoch 196
Validation auc = 0.798122
Epoch 197
Validation auc = 0.779812
Epoch 198
Validation auc = 0.798592
Epoch 199
Loss = 1.2093e-01, PNorm = 95.8752, GNorm = 4.5885, lr_0 = 7.1977e-04
Validation auc = 0.807512
Epoch 200
Validation auc = 0.813146
Epoch 201
Validation auc = 0.813615
Epoch 202
Validation auc = 0.813146
Epoch 203
Validation auc = 0.801878
Epoch 204
Loss = 4.1573e-02, PNorm = 96.2051, GNorm = 4.2423, lr_0 = 7.1362e-04
Validation auc = 0.794366
Epoch 205
Validation auc = 0.788732
Epoch 206
Validation auc = 0.785446
Epoch 207
Validation auc = 0.783568
Epoch 208
Validation auc = 0.778404
Epoch 209
Loss = 4.4187e-02, PNorm = 96.4645, GNorm = 1.7361, lr_0 = 7.0752e-04
Validation auc = 0.788732
Epoch 210
Validation auc = 0.799531
Epoch 211
Validation auc = 0.803756
Epoch 212
Validation auc = 0.804225
Epoch 213
Validation auc = 0.806573
Epoch 214
Loss = 2.2112e-02, PNorm = 96.6638, GNorm = 1.1336, lr_0 = 7.0147e-04
Validation auc = 0.806573
Epoch 215
Validation auc = 0.802347
Epoch 216
Validation auc = 0.804695
Epoch 217
Validation auc = 0.806103
Epoch 218
Validation auc = 0.807981
Epoch 219
Loss = 4.2586e-02, PNorm = 96.8952, GNorm = 2.8277, lr_0 = 6.9548e-04
Validation auc = 0.807981
Epoch 220
Validation auc = 0.801878
Epoch 221
Validation auc = 0.801408
Epoch 222
Validation auc = 0.795775
Epoch 223
Validation auc = 0.792019
Epoch 224
Loss = 7.1730e-02, PNorm = 97.1080, GNorm = 0.4918, lr_0 = 6.8953e-04
Validation auc = 0.790610
Epoch 225
Validation auc = 0.793427
Epoch 226
Validation auc = 0.795775
Epoch 227
Validation auc = 0.796244
Epoch 228
Validation auc = 0.796714
Epoch 229
Loss = 2.3910e-02, PNorm = 97.2764, GNorm = 3.4910, lr_0 = 6.8364e-04
Validation auc = 0.798122
Epoch 230
Validation auc = 0.804695
Epoch 231
Validation auc = 0.806573
Epoch 232
Validation auc = 0.807042
Epoch 233
Validation auc = 0.806573
Epoch 234
Loss = 2.6097e-02, PNorm = 97.4227, GNorm = 1.1338, lr_0 = 6.7779e-04
Validation auc = 0.806103
Epoch 235
Validation auc = 0.807042
Epoch 236
Validation auc = 0.805164
Epoch 237
Validation auc = 0.802347
Epoch 238
Validation auc = 0.803286
Epoch 239
Loss = 5.9348e-02, PNorm = 97.6100, GNorm = 2.4515, lr_0 = 6.7200e-04
Validation auc = 0.803286
Epoch 240
Validation auc = 0.800000
Epoch 241
Validation auc = 0.800000
Epoch 242
Validation auc = 0.795775
Epoch 243
Validation auc = 0.792019
Epoch 244
Loss = 4.1834e-03, PNorm = 97.7648, GNorm = 0.3466, lr_0 = 6.6625e-04
Validation auc = 0.792019
Epoch 245
Validation auc = 0.793897
Epoch 246
Validation auc = 0.796714
Epoch 247
Validation auc = 0.800469
Epoch 248
Validation auc = 0.797653
Epoch 249
Loss = 4.6885e-03, PNorm = 97.9083, GNorm = 0.1974, lr_0 = 6.6056e-04
Validation auc = 0.799061
Epoch 250
Validation auc = 0.795305
Epoch 251
Validation auc = 0.795775
Epoch 252
Validation auc = 0.787324
Epoch 253
Validation auc = 0.785915
Epoch 254
Loss = 2.4646e-02, PNorm = 98.0376, GNorm = 3.3563, lr_0 = 6.5491e-04
Validation auc = 0.789671
Epoch 255
Validation auc = 0.790610
Epoch 256
Validation auc = 0.791080
Epoch 257
Validation auc = 0.790610
Epoch 258
Validation auc = 0.788263
Epoch 259
Loss = 3.4284e-02, PNorm = 98.1968, GNorm = 4.5630, lr_0 = 6.4931e-04
Validation auc = 0.795775
Epoch 260
Validation auc = 0.799531
Epoch 261
Validation auc = 0.803756
Epoch 262
Validation auc = 0.804225
Epoch 263
Validation auc = 0.802817
Epoch 264
Loss = 4.7770e-02, PNorm = 98.3668, GNorm = 3.0574, lr_0 = 6.4376e-04
Validation auc = 0.803756
Epoch 265
Validation auc = 0.806573
Epoch 266
Validation auc = 0.812676
Epoch 267
Validation auc = 0.809390
Epoch 268
Validation auc = 0.807042
Epoch 269
Loss = 1.5061e-02, PNorm = 98.5333, GNorm = 0.1998, lr_0 = 6.3826e-04
Validation auc = 0.810329
Epoch 270
Validation auc = 0.811737
Epoch 271
Validation auc = 0.810798
Epoch 272
Validation auc = 0.811737
Epoch 273
Validation auc = 0.808920
Epoch 274
Loss = 9.1034e-03, PNorm = 98.6868, GNorm = 0.6887, lr_0 = 6.3280e-04
Validation auc = 0.807512
Epoch 275
Validation auc = 0.810329
Epoch 276
Validation auc = 0.809859
Epoch 277
Validation auc = 0.809859
Epoch 278
Validation auc = 0.808920
Epoch 279
Loss = 2.6381e-02, PNorm = 98.7968, GNorm = 1.5597, lr_0 = 6.2739e-04
Validation auc = 0.807512
Epoch 280
Validation auc = 0.801878
Epoch 281
Validation auc = 0.799531
Epoch 282
Validation auc = 0.800000
Epoch 283
Validation auc = 0.801878
Epoch 284
Loss = 1.0026e-02, PNorm = 98.9052, GNorm = 0.9782, lr_0 = 6.2203e-04
Validation auc = 0.804695
Epoch 285
Validation auc = 0.801878
Epoch 286
Validation auc = 0.800469
Epoch 287
Validation auc = 0.799061
Epoch 288
Validation auc = 0.800469
Epoch 289
Loss = 8.7146e-03, PNorm = 99.0122, GNorm = 0.0904, lr_0 = 6.1671e-04
Validation auc = 0.804225
Epoch 290
Validation auc = 0.803286
Epoch 291
Validation auc = 0.803756
Epoch 292
Validation auc = 0.807042
Epoch 293
Validation auc = 0.808920
Epoch 294
Loss = 3.3093e-02, PNorm = 99.1224, GNorm = 0.4519, lr_0 = 6.1144e-04
Validation auc = 0.812676
Epoch 295
Validation auc = 0.813615
Epoch 296
Validation auc = 0.813615
Epoch 297
Validation auc = 0.808920
Epoch 298
Validation auc = 0.807042
Epoch 299
Loss = 1.1497e-02, PNorm = 99.2325, GNorm = 0.9111, lr_0 = 6.0621e-04
Validation auc = 0.804225
Model 0 best validation auc = 0.832864 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.822396
Ensemble test auc = 0.822396
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.15000000000000002, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.15000000000000002, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.15000000000000002, inplace=False)
    (4): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 14,454,301
Moving model to cuda
Epoch 0
Validation auc = 0.653052
Epoch 1
Validation auc = 0.691080
Epoch 2
Validation auc = 0.755869
Epoch 3
Validation auc = 0.770892
Epoch 4
Loss = 1.7493e+00, PNorm = 82.4287, GNorm = 10.6743, lr_0 = 6.5000e-04
Validation auc = 0.774178
Epoch 5
Validation auc = 0.799531
Epoch 6
Validation auc = 0.827700
Epoch 7
Validation auc = 0.847887
Epoch 8
Validation auc = 0.794366
Epoch 9
Loss = 1.3462e+00, PNorm = 82.6904, GNorm = 13.6873, lr_0 = 9.9743e-04
Validation auc = 0.805634
Epoch 10
Validation auc = 0.788263
Epoch 11
Validation auc = 0.794131
Epoch 12
Validation auc = 0.782629
Epoch 13
Validation auc = 0.777934
Epoch 14
Loss = 8.8039e-01, PNorm = 82.9215, GNorm = 13.2678, lr_0 = 9.8890e-04
Validation auc = 0.761972
Epoch 15
Validation auc = 0.759624
Epoch 16
Validation auc = 0.733803
Epoch 17
Validation auc = 0.714554
Epoch 18
Validation auc = 0.721127
Epoch 19
Loss = 7.3266e-01, PNorm = 83.0513, GNorm = 22.5166, lr_0 = 9.8045e-04
Validation auc = 0.722066
Epoch 20
Validation auc = 0.772300
Epoch 21
Validation auc = 0.774648
Epoch 22
Validation auc = 0.779343
Epoch 23
Validation auc = 0.807981
Epoch 24
Loss = 6.8984e-01, PNorm = 83.1309, GNorm = 27.8051, lr_0 = 9.7207e-04
Validation auc = 0.795775
Epoch 25
Validation auc = 0.775117
Epoch 26
Validation auc = 0.774648
Epoch 27
Validation auc = 0.781221
Epoch 28
Validation auc = 0.781690
Epoch 29
Loss = 7.6892e-01, PNorm = 83.1960, GNorm = 15.6712, lr_0 = 9.6376e-04
Validation auc = 0.771831
Epoch 30
Validation auc = 0.773709
Epoch 31
Validation auc = 0.791080
Epoch 32
Validation auc = 0.785446
Epoch 33
Validation auc = 0.753991
Epoch 34
Loss = 5.2519e-01, PNorm = 83.2660, GNorm = 17.7466, lr_0 = 9.5552e-04
Validation auc = 0.732864
Epoch 35
Validation auc = 0.718779
Epoch 36
Validation auc = 0.714554
Epoch 37
Validation auc = 0.703756
Epoch 38
Validation auc = 0.735681
Epoch 39
Loss = 7.1465e-01, PNorm = 83.3572, GNorm = 23.9532, lr_0 = 9.4735e-04
Validation auc = 0.742723
Epoch 40
Validation auc = 0.739437
Epoch 41
Validation auc = 0.738498
Epoch 42
Validation auc = 0.723474
Epoch 43
Validation auc = 0.726761
Epoch 44
Loss = 8.5565e-01, PNorm = 83.4991, GNorm = 17.2452, lr_0 = 9.3925e-04
Validation auc = 0.734272
Epoch 45
Validation auc = 0.742254
Epoch 46
Validation auc = 0.720188
Epoch 47
Validation auc = 0.725352
Epoch 48
Validation auc = 0.715023
Epoch 49
Loss = 6.1427e-01, PNorm = 83.6874, GNorm = 13.0332, lr_0 = 9.3122e-04
Validation auc = 0.719249
Epoch 50
Validation auc = 0.746479
Epoch 51
Validation auc = 0.743192
Epoch 52
Validation auc = 0.740376
Epoch 53
Validation auc = 0.744131
Epoch 54
Loss = 4.1066e-01, PNorm = 83.9071, GNorm = 10.6098, lr_0 = 9.2326e-04
Validation auc = 0.762441
Epoch 55
Validation auc = 0.784977
Epoch 56
Validation auc = 0.784038
Epoch 57
Validation auc = 0.779812
Epoch 58
Validation auc = 0.758685
Epoch 59
Loss = 3.9407e-01, PNorm = 84.1500, GNorm = 9.2060, lr_0 = 9.1537e-04
Validation auc = 0.775587
Epoch 60
Validation auc = 0.763380
Epoch 61
Validation auc = 0.762911
Epoch 62
Validation auc = 0.735211
Epoch 63
Validation auc = 0.740845
Epoch 64
Loss = 3.6477e-01, PNorm = 84.4026, GNorm = 10.8582, lr_0 = 9.0754e-04
Validation auc = 0.767606
Epoch 65
Validation auc = 0.774648
Epoch 66
Validation auc = 0.763380
Epoch 67
Validation auc = 0.746948
Epoch 68
Validation auc = 0.769953
Epoch 69
Loss = 5.3166e-01, PNorm = 84.6222, GNorm = 20.5608, lr_0 = 8.9978e-04
Validation auc = 0.774178
Epoch 70
Validation auc = 0.777934
Epoch 71
Validation auc = 0.780282
Epoch 72
Validation auc = 0.789202
Epoch 73
Validation auc = 0.787324
Epoch 74
Loss = 4.4480e-01, PNorm = 84.8484, GNorm = 13.1642, lr_0 = 8.9209e-04
Validation auc = 0.782629
Epoch 75
Validation auc = 0.790141
Epoch 76
Validation auc = 0.768075
Epoch 77
Validation auc = 0.756808
Epoch 78
Validation auc = 0.769484
Epoch 79
Loss = 4.8672e-01, PNorm = 85.1513, GNorm = 17.7891, lr_0 = 8.8447e-04
Validation auc = 0.759624
Epoch 80
Validation auc = 0.754460
Epoch 81
Validation auc = 0.769484
Epoch 82
Validation auc = 0.781690
Epoch 83
Validation auc = 0.780282
Epoch 84
Loss = 3.4389e-01, PNorm = 85.4173, GNorm = 6.0748, lr_0 = 8.7691e-04
Validation auc = 0.776526
Epoch 85
Validation auc = 0.781221
Epoch 86
Validation auc = 0.784507
Epoch 87
Validation auc = 0.768075
Epoch 88
Validation auc = 0.766667
Epoch 89
Loss = 3.0620e-01, PNorm = 85.7051, GNorm = 5.8991, lr_0 = 8.6941e-04
Validation auc = 0.780282
Epoch 90
Validation auc = 0.784038
Epoch 91
Validation auc = 0.784977
Epoch 92
Validation auc = 0.778873
Epoch 93
Validation auc = 0.782629
Epoch 94
Loss = 1.7226e-01, PNorm = 86.0260, GNorm = 7.0460, lr_0 = 8.6198e-04
Validation auc = 0.793427
Epoch 95
Validation auc = 0.794836
Epoch 96
Validation auc = 0.805164
Epoch 97
Validation auc = 0.795305
Epoch 98
Validation auc = 0.808451
Epoch 99
Loss = 1.1424e-01, PNorm = 86.3511, GNorm = 5.3147, lr_0 = 8.5461e-04
Validation auc = 0.814554
Epoch 100
Validation auc = 0.807042
Epoch 101
Validation auc = 0.800000
Epoch 102
Validation auc = 0.810329
Epoch 103
Validation auc = 0.816901
Epoch 104
Loss = 3.9319e-01, PNorm = 86.6951, GNorm = 16.5621, lr_0 = 8.4730e-04
Validation auc = 0.825352
Epoch 105
Validation auc = 0.797653
Epoch 106
Validation auc = 0.784038
Epoch 107
Validation auc = 0.801878
Epoch 108
Validation auc = 0.784507
Epoch 109
Loss = 2.1547e-01, PNorm = 87.0533, GNorm = 9.4339, lr_0 = 8.4006e-04
Validation auc = 0.770423
Epoch 110
Validation auc = 0.783099
Epoch 111
Validation auc = 0.801408
Epoch 112
Validation auc = 0.809390
Epoch 113
Validation auc = 0.825352
Epoch 114
Loss = 2.4042e-01, PNorm = 87.3877, GNorm = 5.0092, lr_0 = 8.3288e-04
Validation auc = 0.826291
Epoch 115
Validation auc = 0.821127
Epoch 116
Validation auc = 0.810329
Epoch 117
Validation auc = 0.797653
Epoch 118
Validation auc = 0.788732
Epoch 119
Loss = 8.6114e-02, PNorm = 87.6914, GNorm = 4.4747, lr_0 = 8.2576e-04
Validation auc = 0.783568
Epoch 120
Validation auc = 0.785446
Epoch 121
Validation auc = 0.797653
Epoch 122
Validation auc = 0.804695
Epoch 123
Validation auc = 0.805634
Epoch 124
Loss = 1.9571e-01, PNorm = 87.9820, GNorm = 7.3680, lr_0 = 8.1870e-04
Validation auc = 0.812676
Epoch 125
Validation auc = 0.809859
Epoch 126
Validation auc = 0.803756
Epoch 127
Validation auc = 0.787324
Epoch 128
Validation auc = 0.781690
Epoch 129
Loss = 9.0484e-02, PNorm = 88.3042, GNorm = 7.6839, lr_0 = 8.1170e-04
Validation auc = 0.777934
Epoch 130
Validation auc = 0.784038
Epoch 131
Validation auc = 0.794836
Epoch 132
Validation auc = 0.795775
Epoch 133
Validation auc = 0.807042
Epoch 134
Loss = 4.6573e-02, PNorm = 88.6595, GNorm = 3.1075, lr_0 = 8.0476e-04
Validation auc = 0.812207
Epoch 135
Validation auc = 0.814554
Epoch 136
Validation auc = 0.815023
Epoch 137
Validation auc = 0.813146
Epoch 138
Validation auc = 0.808920
Epoch 139
Loss = 7.5040e-02, PNorm = 89.0142, GNorm = 2.2870, lr_0 = 7.9788e-04
Validation auc = 0.797653
Epoch 140
Validation auc = 0.788263
Epoch 141
Validation auc = 0.782160
Epoch 142
Validation auc = 0.780751
Epoch 143
Validation auc = 0.774178
Epoch 144
Loss = 1.3920e-01, PNorm = 89.3479, GNorm = 4.1764, lr_0 = 7.9106e-04
Validation auc = 0.773239
Epoch 145
Validation auc = 0.779812
Epoch 146
Validation auc = 0.792488
Epoch 147
Validation auc = 0.797653
Epoch 148
Validation auc = 0.795305
Epoch 149
Loss = 5.8053e-02, PNorm = 89.6652, GNorm = 0.7775, lr_0 = 7.8430e-04
Validation auc = 0.789202
Epoch 150
Validation auc = 0.785915
Epoch 151
Validation auc = 0.788732
Epoch 152
Validation auc = 0.789202
Epoch 153
Validation auc = 0.773239
Epoch 154
Loss = 9.5197e-02, PNorm = 89.9722, GNorm = 6.6954, lr_0 = 7.7759e-04
Validation auc = 0.769014
Epoch 155
Validation auc = 0.781690
Epoch 156
Validation auc = 0.782160
Epoch 157
Validation auc = 0.774648
Epoch 158
Validation auc = 0.778404
Epoch 159
Loss = 8.4680e-02, PNorm = 90.2540, GNorm = 4.0227, lr_0 = 7.7095e-04
Validation auc = 0.775587
Epoch 160
Validation auc = 0.787793
Epoch 161
Validation auc = 0.795775
Epoch 162
Validation auc = 0.796714
Epoch 163
Validation auc = 0.800939
Epoch 164
Loss = 6.1059e-02, PNorm = 90.5581, GNorm = 3.4508, lr_0 = 7.6436e-04
Validation auc = 0.795305
Epoch 165
Validation auc = 0.795775
Epoch 166
Validation auc = 0.805164
Epoch 167
Validation auc = 0.814085
Epoch 168
Validation auc = 0.813146
Epoch 169
Loss = 8.7891e-02, PNorm = 90.8102, GNorm = 3.6097, lr_0 = 7.5782e-04
Validation auc = 0.813615
Epoch 170
Validation auc = 0.810329
Epoch 171
Validation auc = 0.807512
Epoch 172
Validation auc = 0.804695
Epoch 173
Validation auc = 0.807042
Epoch 174
Loss = 9.5748e-02, PNorm = 91.0480, GNorm = 3.2103, lr_0 = 7.5134e-04
Validation auc = 0.809859
Epoch 175
Validation auc = 0.807981
Epoch 176
Validation auc = 0.800000
Epoch 177
Validation auc = 0.804695
Epoch 178
Validation auc = 0.804695
Epoch 179
Loss = 7.5923e-02, PNorm = 91.2870, GNorm = 4.1328, lr_0 = 7.4492e-04
Validation auc = 0.804695
Epoch 180
Validation auc = 0.802347
Epoch 181
Validation auc = 0.807042
Epoch 182
Validation auc = 0.798122
Epoch 183
Validation auc = 0.791080
Epoch 184
Loss = 1.6075e-01, PNorm = 91.6357, GNorm = 8.3369, lr_0 = 7.3855e-04
Validation auc = 0.802347
Epoch 185
Validation auc = 0.792958
Epoch 186
Validation auc = 0.789671
Epoch 187
Validation auc = 0.783099
Epoch 188
Validation auc = 0.779343
Epoch 189
Loss = 3.6611e-02, PNorm = 91.9822, GNorm = 3.0032, lr_0 = 7.3224e-04
Validation auc = 0.778404
Epoch 190
Validation auc = 0.775587
Epoch 191
Validation auc = 0.774648
Epoch 192
Validation auc = 0.779343
Epoch 193
Validation auc = 0.788263
Epoch 194
Loss = 3.4496e-02, PNorm = 92.2968, GNorm = 2.8680, lr_0 = 7.2598e-04
Validation auc = 0.792488
Epoch 195
Validation auc = 0.800000
Epoch 196
Validation auc = 0.801408
Epoch 197
Validation auc = 0.796714
Epoch 198
Validation auc = 0.799061
Epoch 199
Loss = 1.2043e-01, PNorm = 92.5709, GNorm = 5.2917, lr_0 = 7.1977e-04
Validation auc = 0.800939
Epoch 200
Validation auc = 0.798122
Epoch 201
Validation auc = 0.798592
Epoch 202
Validation auc = 0.796714
Epoch 203
Validation auc = 0.793897
Epoch 204
Loss = 1.0333e-01, PNorm = 92.7704, GNorm = 2.9872, lr_0 = 7.1362e-04
Validation auc = 0.796714
Epoch 205
Validation auc = 0.799061
Epoch 206
Validation auc = 0.799531
Epoch 207
Validation auc = 0.801878
Epoch 208
Validation auc = 0.807512
Epoch 209
Loss = 6.7328e-02, PNorm = 93.0128, GNorm = 2.5315, lr_0 = 7.0752e-04
Validation auc = 0.814085
Epoch 210
Validation auc = 0.807981
Epoch 211
Validation auc = 0.811737
Epoch 212
Validation auc = 0.809390
Epoch 213
Validation auc = 0.809390
Epoch 214
Loss = 6.7888e-02, PNorm = 93.2969, GNorm = 1.8661, lr_0 = 7.0147e-04
Validation auc = 0.817371
Epoch 215
Validation auc = 0.824413
Epoch 216
Validation auc = 0.818779
Epoch 217
Validation auc = 0.810329
Epoch 218
Validation auc = 0.804695
Epoch 219
Loss = 8.5437e-02, PNorm = 93.5401, GNorm = 3.4244, lr_0 = 6.9548e-04
Validation auc = 0.799531
Epoch 220
Validation auc = 0.797653
Epoch 221
Validation auc = 0.800000
Epoch 222
Validation auc = 0.798592
Epoch 223
Validation auc = 0.798592
Epoch 224
Loss = 4.3735e-02, PNorm = 93.7743, GNorm = 1.9331, lr_0 = 6.8953e-04
Validation auc = 0.796714
Epoch 225
Validation auc = 0.793427
Epoch 226
Validation auc = 0.794366
Epoch 227
Validation auc = 0.790610
Epoch 228
Validation auc = 0.793897
Epoch 229
Loss = 2.7364e-02, PNorm = 93.9465, GNorm = 1.5384, lr_0 = 6.8364e-04
Validation auc = 0.797183
Epoch 230
Validation auc = 0.798592
Epoch 231
Validation auc = 0.804225
Epoch 232
Validation auc = 0.803286
Epoch 233
Validation auc = 0.802817
Epoch 234
Loss = 3.1971e-02, PNorm = 94.1055, GNorm = 1.9011, lr_0 = 6.7779e-04
Validation auc = 0.800469
Epoch 235
Validation auc = 0.799061
Epoch 236
Validation auc = 0.796244
Epoch 237
Validation auc = 0.800939
Epoch 238
Validation auc = 0.811737
Epoch 239
Loss = 1.5520e-02, PNorm = 94.2590, GNorm = 1.1512, lr_0 = 6.7200e-04
Validation auc = 0.810329
Epoch 240
Validation auc = 0.807981
Epoch 241
Validation auc = 0.806573
Epoch 242
Validation auc = 0.807042
Epoch 243
Validation auc = 0.806573
Epoch 244
Loss = 4.4980e-02, PNorm = 94.4229, GNorm = 2.3390, lr_0 = 6.6625e-04
Validation auc = 0.807981
Epoch 245
Validation auc = 0.805634
Epoch 246
Validation auc = 0.806573
Epoch 247
Validation auc = 0.803286
Epoch 248
Validation auc = 0.805634
Epoch 249
Loss = 1.3653e-02, PNorm = 94.5752, GNorm = 1.1272, lr_0 = 6.6056e-04
Validation auc = 0.801408
Epoch 250
Validation auc = 0.801408
Epoch 251
Validation auc = 0.797653
Epoch 252
Validation auc = 0.793427
Epoch 253
Validation auc = 0.792488
Epoch 254
Loss = 4.0493e-02, PNorm = 94.7597, GNorm = 1.5313, lr_0 = 6.5491e-04
Validation auc = 0.790610
Epoch 255
Validation auc = 0.790141
Epoch 256
Validation auc = 0.791080
Epoch 257
Validation auc = 0.789671
Epoch 258
Validation auc = 0.791549
Epoch 259
Loss = 2.1757e-02, PNorm = 94.9456, GNorm = 0.6308, lr_0 = 6.4931e-04
Validation auc = 0.791080
Epoch 260
Validation auc = 0.792488
Epoch 261
Validation auc = 0.793427
Epoch 262
Validation auc = 0.794836
Epoch 263
Validation auc = 0.794366
Epoch 264
Loss = 1.6228e-02, PNorm = 95.1232, GNorm = 2.2078, lr_0 = 6.4376e-04
Validation auc = 0.795305
Epoch 265
Validation auc = 0.792488
Epoch 266
Validation auc = 0.790141
Epoch 267
Validation auc = 0.786854
Epoch 268
Validation auc = 0.781690
Epoch 269
Loss = 5.0817e-02, PNorm = 95.2971, GNorm = 0.5562, lr_0 = 6.3826e-04
Validation auc = 0.782160
Epoch 270
Validation auc = 0.791549
Epoch 271
Validation auc = 0.793427
Epoch 272
Validation auc = 0.796714
Epoch 273
Validation auc = 0.794836
Epoch 274
Loss = 3.0140e-02, PNorm = 95.4570, GNorm = 5.9625, lr_0 = 6.3280e-04
Validation auc = 0.800939
Epoch 275
Validation auc = 0.804225
Epoch 276
Validation auc = 0.800000
Epoch 277
Validation auc = 0.791080
Epoch 278
Validation auc = 0.783568
Epoch 279
Loss = 3.6431e-02, PNorm = 95.6853, GNorm = 2.6441, lr_0 = 6.2739e-04
Validation auc = 0.784977
Epoch 280
Validation auc = 0.785915
Epoch 281
Validation auc = 0.792019
Epoch 282
Validation auc = 0.791080
Epoch 283
Validation auc = 0.794836
Epoch 284
Loss = 9.3685e-03, PNorm = 95.9064, GNorm = 1.2088, lr_0 = 6.2203e-04
Validation auc = 0.794366
Epoch 285
Validation auc = 0.791549
Epoch 286
Validation auc = 0.789671
Epoch 287
Validation auc = 0.789202
Epoch 288
Validation auc = 0.793427
Epoch 289
Loss = 1.2705e-02, PNorm = 96.0743, GNorm = 0.8317, lr_0 = 6.1671e-04
Validation auc = 0.792019
Epoch 290
Validation auc = 0.791549
Epoch 291
Validation auc = 0.792958
Epoch 292
Validation auc = 0.790610
Epoch 293
Validation auc = 0.780282
Epoch 294
Loss = 4.6890e-02, PNorm = 96.2558, GNorm = 2.3749, lr_0 = 6.1144e-04
Validation auc = 0.781221
Epoch 295
Validation auc = 0.777465
Epoch 296
Validation auc = 0.779343
Epoch 297
Validation auc = 0.786854
Epoch 298
Validation auc = 0.791549
Epoch 299
Loss = 9.6166e-03, PNorm = 96.4865, GNorm = 1.2158, lr_0 = 6.0621e-04
Validation auc = 0.792019
Model 0 best validation auc = 0.847887 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.835039
Ensemble test auc = 0.835039
10-fold cross validation
	Seed 0 ==> test auc = 0.804335
	Seed 1 ==> test auc = 0.824804
	Seed 2 ==> test auc = 0.778447
	Seed 3 ==> test auc = 0.775436
	Seed 4 ==> test auc = 0.802529
	Seed 5 ==> test auc = 0.860325
	Seed 6 ==> test auc = 0.878387
	Seed 7 ==> test auc = 0.864539
	Seed 8 ==> test auc = 0.822396
	Seed 9 ==> test auc = 0.835039
Overall test auc = 0.824624 +/- 0.033650
Elapsed time = 0:04:45
